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Building  More  Efficient  Processes  and  Bridging  Communication  Gaps 

L.  Matthew  Foster,  Michael  Flory,  Ninghao  Jiang,  and  William  McNavage 


CNA 


This  report  contains  the  Distributed  Engineering  Plant  (DEP)  Bottom  Up  Review 
(BUR)  for  fiscal  year  (FY)  08.  We  examined  tools  that  can  trace  Interoperability 
Certification  Committee  (ICC)  requirements  through  the  DEP  testing  and  analysis 
processes.  This  traceability  is  a  required  outcome  of  the  combined  ICC  and  Data 
Management  &  Analysis  (DM&A)  Rapid  Improvement  Event  (RIE);  it  should  improve 
the  overall  quality  of  the  testing  by  improving  the  documentation  that  seeds  the 
overall  DEP  process.  We  also  suggest  roles  for  key  stakeholders  based  on  the 
proposed  process  improvements  that  were  developed  during  the  ICC  and  DM&A 
RIE. 
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The  Navy  Distributed  Engineering  Plant  (PEP) 

•  Shore-based  test  bed 

-  Replicates  ESG/CSG  conducting  anti-air  warfare  operations  at  sea 

-  Connects  dispersed  land  based  sites  around  CONUS  via  a  secure  network 

•  Preserves  hardware  in  the  loop 

-  Rea /  Combat  Systems  (Aegis,  ACDS,  SGS/AC,  etc.) 

-  Simulated  tactical  connectivity  (Link-1 1  Link-16,  CEC) 

-  Simulated  Radar  Input 

•  Tests  interoperability  of  combat  systems 

-  The  condition  achieved  among  communications-electronics  systems  .  when 
information  or  services  can  be  exchanged  directly  and  satisfactorily  -  DoD 
Dictionary  of  Military  and  Assoc.  Terms 

-  Main  focus  is  testing  systems’  abilities  to  maintain  a  Common  Air  Tactical  Picture 

•  Scripted  air  defense  combat  scenario  synchronously  fed  to  each  land-based 
combat  system  site 

•  Significant  cost  savings  over  live  exercises 

•  Provides  controlled  environment  to  investigate  combat  system 
interoperability 

CNA 


The  Navy  DEP  is  a  shore-based  testing  facility  that  simulates  Expeditionary  Strike 
Groups  (ESGs)  and  Carrier  Strike  Groups  (CSGs)  conducting  anti-air  warfare 
operations  at  sea.  DEP  connects  distributed  land-based  sites  throughout  the  United 
States  using  secure  networks.  One  major  advantage  of  DEP  is  the  preservation  of 
“hardware  in  the  loop”;  real  combat  systems  are  tested  by  examining  their 
responses  to  simulated  tactical  radar  inputs.  The  DEP  environment  can  test  the 
interoperability  of  these  combat  systems  and  the  ability  of  the  ESG/CSG  to  maintain 
a  Common  Air  Tactical  Picture. 

Operators  have  minimal  interaction  with  the  combat  systems  during  DEP  test 
execution.  This  minimal  interaction  will  test  the  actual  combat  system’s  ability  to 
react  as  part  of  an  entire  ESG/CSG.  The  scenarios  used  within  DEP  tests  are 
scripted  and  synchronously  fed  to  each  land-based  site.  The  combat  systems  are 
connected  through  shared  simulated  tactical  communication  networks  (i.e.,  Link-11, 
Link-16,  Cooperative  Engagement  Capability  (CEC)).  The  combat  systems  operate 
as  if  they  were  collocated  in  an  ESG/CSG  at  sea.  The  combat  systems’  reactions 
are  recorded  while  the  scripted  scenario  is  running.  The  recorded  combat  system 
data  are  later  analyzed  to  identify  interoperability  issues  with  the  system(s)  under 
test. 
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(continued) 

The  DEP  environment  can  provide  more  controlled  conditions  over  a  live  event  in 
order  to  investigate  the  interoperability  of  combat  systems  within  an  ESG/CSG.  One 
of  the  largest  benefits  of  the  DEP  is  that  combat  system  interoperability  can  be 
tested  in  a  simulated,  land-based  environment,  dramatically  reducing  the  cost  of 
conducting  these  tests  using  live  assets.  Additionally,  the  DEP  can  test 
developmental  versions  of  the  combat  system  software  in  the  presence  of  other 
networked  combat  systems.  These  software  loads  can  be  tested  at  an  early  enough 
developmental  stage  such  that  developers  can  detect,  and  potentially  fix  or  provide 
work-arounds  for  issues  that  may  prevent  and  prohibit  a  strike  force  from  being 
interoperable. 
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The  DEP  is  composed  of  various  combat  system  laboratories  located  across  the 
continental  United  States  (CONUS).  Most  DEP  sites  are  located  on  the  East  or 
West  Coasts.  Each  testing  site  can  house  a  single  or  multiple  combat  systems.  The 
DEP  operations  center  (DOC),  located  at  the  Integrated  Warfare  Systems 
Laboratory  at  the  Naval  Surface  Warfare  Center  Dahlgren  Division  (NSWCDD)  in 
Dahlgren,  Virginia,  is  the  centralized  location  and  hub  where  all  tests  are  executed. 
The  DOC  controls  all  scenario  feeds  to  the  various  participating  combat  system 
laboratories.  Also,  the  DOC  monitors  network-wide  connectivity  during  test 
execution.  During  a  DEP  event  focused  on  combat  system  interoperability,  multiple 
(4  to  5)  combat  systems  will  typically  participate,  in  various  configurations  specific  to 
the  test  objectives  and  requirements  that  must  be  satisfied. 
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CNA 


The  Navy  DEP  program  exists  under  the  Naval  Sea  Systems  Command  (NAVSEA) 
05W42  code.  The  DEP  program,  led  by  the  DEP  Program  Manager,  is  composed  of 
three  distinct  functional  areas:  Engineering  &  Operations  (E&O),  Test  Planning  & 
Execution  (TP&E),  and  Data  Management  &  Analysis  (DM&A). 

The  E&O  functional  area  is  responsible  for  maintaining  network  fidelity  and 
improvements,  DOC  management,  scenario  drivers,  and  site  management  for  each 
DEP  testing  event.  Also,  E&O  is  responsible  for  ensuring  that  each  testing  site  has 
the  proper  security  accreditation  in  order  to  establish  connectivity  to  the  DEP 
network  and  participate  in  an  interoperability  assessment  test. 

The  TP&E  functional  area  plans  each  DEP  test  event  in  addition  to  scheduling  the 
laboratories  needed  to  execute  the  events.  TP&E  also  hosts  the  Test  Director  (TD), 
who  is  responsible  for  test  execution.  TP&E  works  closely  with  E&O  to  develop  and 
maintain  testing  scenarios  that  examine  various  interoperability  issues. 

The  DM&A  functional  area  is  responsible  for  the  management  of  data  recorded  at 
participating  combat  system  laboratories  and  the  DOC  to  ensure  that  the  data  set  is 
complete,  valid,  and  suitable  for  post-event  analysis.  Also  DM&A  is  responsible  for 
the  completion  of  post-event  analyses  to  support  interoperability  assessment. 

Finally,  DM&A  assembles  all  valid  interoperability  issues  discovered  both  during  test 
execution  and  post-event  analysis  and  presents  those  findings  to  the  DEP  PM  and 
ICC. 
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(continued) 


The  DEP  PM  is  CNA’s  government  sponsor,  and  the  PM  historically  has  tasked 
CNA  to  support  the  DM&A  functional  area.  CNA  has  provided  support  in  the  form  of 
analysis  focused  on  interoperability  metrics  [1],  In  addition  to  this  role,  CNA  has 
frequently  been  involved  in  test  planning  by  participating  in  Test  Planning  Working 
Groups  and  by  reviewing  test  procedures.  Further,  CNA  has  been  present  for  the 
execution  of  numerous  DEP  interoperability  tests. 

While  the  DEP  functional  areas  are  responsible  for  interoperability  assessment 
testing  execution  and  analysis,  the  ICC  is  the  driving  force  that  determines  which 
systems  should  be  brought  into  the  DEP  to  be  tested.  The  ICC  is  also  responsible 
for  fusing  analysis  provided  from  DEP  test  results  into  a  high-level  interoperability 
assessment.  The  ICC  works  with  TP&E  to  determine  the  necessary  combat 
systems  needed  for  test  execution. 
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Interoperability  Certification  Committee  (ICC) 

•  Separate  organization  within  NAVSEA 

CNA  also  directly  supports  ICC  with  analysis  and  assessment 

•  Functional  areas  analogous  to  DEP 

ICC  Assessment  ->  Data  Management  &  Analysis  (DM&A) 

ICC  Test  Management  ->  Test  Planning  &  Execution  (TP&E) 

•  Provides  combat  system  interoperability  certification  assessment 
recommendations  to  the  Fleet 

Provides  data  from  interoperability  assessments  to  developers  to  drive 
software  changes 

Identifies  Tactics,  Techniques,  and  Procedures 

•  ICC  uses  results  from  DEP  tests  as  one  of  many  sources  of  data 


CNA 


The  Interoperability  Certification  Committee  exists  as  an  organization  within 
NAVSEA  separate  from  the  DEP.  ICC  membership  spans  various  government  and 
contractor  organizations,  and  its  primary  role  is  to  determine  whether  a  combat 
system  and  its  related  subsystems  have  met  various  functional  requirements 
necessary  to  interoperate  with  an  ESG/CSG.  CNA  also  directly  supports  the  ICC 
with  analysis  and  assessment.  The  ICC  is  supported  by  eight  functional  leads  with 
an  overall  lead  or  chairperson.  The  ICC  functional  areas  are  the  following: 

•Interoperability  Certification  Requirements 

•Interoperability  Assessment 

•Interoperability  Test  Management 

•Systems  Engineering 

•Fleet  Operations 

•Fleet  Reporting 

•Force  Safety 

•Navy  Link  Certification. 
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(continued) 

Many  of  these  ICC  functional  areas  are  analogous  to  some  of  the  DEP  functional 
areas.  The  ICC  Assessment’s  counterpart  within  DEP  is  DM&A,  and  the  ICC  Test 
Management’s  is  TP&E.  The  ICC  does  not  conduct  the  actual  testing;  they  request 
that  interoperability  assessment  tests  be  performed  within  the  DEP  environment 
and  other  testing  programs,  such  as  Warfare  Systems  Integration  and 
Interoperability  Testing  (WSIIT)  and  Navy  Center  for  Tactical  Systems 
Interoperability  (NCTSI). 

The  goal  of  the  ICC  is  to  enable  an  all-inclusive  approach  to  understanding  the 
interoperability  requirements  of  Navy  systems,  defining  certification  criteria  based 
on  operational,  performance,  and  programmatic  requirements.  Also,  the  ICC  links 
the  certification  process  to  systems  engineering  efforts  and  also  the  ICC  assesses 
the  warfare  system  interoperability  performance  and  risk  from  the  technical  and 
operational  perspective,  including  effects  at  the  Force  level.  Finally,  the  ICC  works 
with  the  Capabilities  and  Limitations  (C&L)  program  to  ensure  trainers,  sailors,  and 
staffs  are  aware  of  the  interoperability  characteristics  of  the  various  platforms  and 
collections  of  platforms.  It  is  the  committee’s  mission  to  ensure  work-arounds  and 
that  the  tactics,  techniques  and  procedures  (TTPs)  regarding  these  are  distributed 
and  understood  [2]. 

Since  the  ICC  was  stood  up  in  2005,  the  interface  between  the  ICC  and  the  DEP 
functional  areas  has  faced  various  organizational  and  communication  challenges. 
Many  of  these  issues  have  resulted  in  tests  that  were  not  executed  or  analytic 
results  that  did  not  align  with  the  expectations  of  the  ICC.  CNA  has  observed  many 
of  these  issues  when  working  with  both  the  DEP  and  the  ICC,  and  we  have  been 
tasked  to  resolve  known  communication  gaps  and  offer  a  suite  of  tools  to  help 
improve  the  overall  interoperability  assessment  test  process.  We  will  elaborate 
more  on  specific  issues  later  in  this  document. 
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CNA  original  tasking 


■  DEP  Program  Manager  tasked  CNA  to  lead  the  DEP  Bottom  Up 
Review  (BUR)  as  outlined  in  the  FY08  DEP  Strategic  Plan. 

The  Objective  is  to  review  the  methodology  used  to  determine  event 
objectives  for  interoperability  assessment  tests. 

The  Desired  Outcome  is  to  generate  sound  process  improvement 
recommendations  to  be  carried  forward  into  future  DEP  testing 
(planning,  engineering,  data  management,  and  analysis). 

■  However,  CNA  found  the  intended  scope  of  the  BUR  was  much 
broader. 

•  CNA  is  uniquely  positioned  to  lead  process  review 
Hybrid  approach 

■  Active  in  multiple  divisions  of  DEP/ICC 

-  Participated  in  ICC,  planning,  testing,  and  analysis  for  many  years 

CNA 


The  DEP  Program  Manager  in  NAVSEA  tasked  CNA  to  support  the  DEP  BUR  as 
outlined  in  the  FY08  DEP  Strategic  Plan.  The  initial  DEP  BUR  objective  was  to 
review  the  methodologies  used  to  determine  DEP  test  objectives.  However,  CNA 
found  the  intended  scope  of  the  BUR  was  much  broader  and  required  creating 
traceability  within  the  entire  testing  process  from  the  initial  ICC  test  request  to  the 
final  brief  delivered  by  DEP  DM&A  to  the  ICC  and  DEP  Program  Manager.  The  ICC 
only  provides  test  objectives  to  the  DEP  prior  to  test  planning  rather  than  providing 
the  full  scope  of  both  test  objectives  and  the  requirements  they  satisfy.  This  practice 
has  caused  several  disconnects  in  both  the  test  and  analysis  process  for  various 
interoperability  assessment  tests  that  resulted  in  analysis  that  did  not  support  the 
needs  of  the  ICC. 

The  DEP  test  objectives  are  proposed  by  the  ICC  Test  Management  and  handed 
over  to  TP&E  for  event  planning  through  a  test  planning  guide.  This  test  planning 
guide  is  currently  an  informal  document  that  is  passed  from  the  ICC  Test 
Management  to  the  TP&E  lead  prior  to  the  first  test  event  planning  meeting.  This 
planning  guide  represents  the  first  step  in  establishing  organizational  traceability. 
This  test  planning  guide  needs  to  directly  connect  the  ICC  requirements  and 
motivations  to  the  proposed  test  objectives.  Then  with  a  developed  test  planning 
guide,  the  DEP  functional  areas  can  connect  DEP  planning,  engineering,  and  data 
analysis  to  the  initial  ICC  test  planning  guide  and  objectives. 
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(continued) 

The  final  BUR  goals  are  to  generate  process  improvement  recommendations  that 
will  help  create  and  restore  organizational  traceability  such  as  with  a  formal  ICC  test 
planning  guide.  CNA  is  uniquely  positioned  to  lead  the  BUR  process  because 
historically  CNA  has  worked  with  each  DEP  functional  area  as  well  as  had  direct 
support  and  insight  into  the  ICC.  CNA’s  ability  to  cross  functional  areas  will  help  in 
creating  process  improvements  that  will  allow  the  DEP  and  ICC  to  be  successful  in 
future  testing  events. 
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What  “data”  do  we  have? 

•  DEP  conducted  several  process  improvement  events  using  the  Lean  Six 
Sigma  program 

ICC+DEP  Rapid  Improvement  Event  (RIE) 

•  Outlined  Future  State  of  ICC-DEP  testing  process 
DM&A  Value  Stream  Analysis 
Test  Observation  Report  RIE 

•  CNA  participated  in  these  events  or  obtained  available  outputs 

Process  flow  charts 

Responsibility,  Accountability,  Consultation,  Information  charts 
Action  items 

•  Reviewed  existing  test  documentation  and  analysis  methodologies 

Test  Planning  Guide  (ICC) 

Test  Procedures  (TP&E) 

Data  Management  and  Analysis  Plan  (DM&A) 

Scheduling  documents 
TCB  Charter 

CNA 


As  we  discussed  earlier,  the  FY08  DEP  strategic  plan  calls  for  a  review  of  the 
methodology  used  to  translate  high-level  ICC  requirements  into  test  objectives  that 
will  be  tested  within  the  DEP.  CNA  was  tasked  to  provide  a  method  for  tracing  test 
objectives  back  to  the  original  high-level  ICC  requirements.  Several  process 
improvement  events  fed  into  the  DEP  BUR.  First,  the  DEP  DM&A  functional  area 
conducted  a  Value  Stream  Analysis  (VSA),  a  process  improvement  and 
streamlining  event,  in  mid-2007  to  determine  which  areas  of  the  testing  process 
could  be  improved.  DM&A  concluded  from  the  VSA  event  that  several  additional 
streamlining  events  should  occur  with  various  stakeholders  from  across  the  DEP 
organization.  As  a  result,  the  ICC  and  DM&A  conducted  a  Rapid  Improvement 
Event  (RIE).  This  RIE  critically  reviewed  the  current  ICC  processes  that  seed  DEP 
interoperability  assessment  tests  and  also  reexamined  steps  within  the  actual  DEP 
testing  process.  These  process  improvements  stemmed  from  a  Navy-adopted 
program  called  Lean  Six  Sigma.  Lean  Six  Sigma  combines  two  improvement 
trends:  making  work  better  and  making  work  faster.  The  idea  behind  Lean  Six 
Sigma  is  identifying  gaps  in  processes  and  closing  those  gaps  through 
communication  and  collaboration  within  the  organization  [3], 

During  the  RIE,  the  ICC,  with  input  from  the  DEP  Functional  Leads,  created  a 
revised  testing  process  denoted  as  the  Future  State.  The  proposed  process  Future 
State  was  built  off  of  input  and  recommendations  made  during  the  RIE  execution  by 
DEP  functional  leads,  ICC  members,  and  the  DEP  program  manager.  The  Future 
State,  which  will  be  discussed  in  more  detail  later,  outlines  the  work  flow 
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(continued) 

from  the  initial  ICC  test  request  through  the  delivery  of  test  results  by  DEP 
personnel  to  the  ICC.  The  ICC/DM&A  RIE  offered  a  framework  for  CNA  to  conduct 
the  BUR.  The  RIE  outcomes  provide  CNA  with  a  set  of  guidelines  that  have  been 
agreed  upon  by  both  the  ICC  membership  and  DEP  functional  areas. 

In  an  attempt  to  leverage  existing  testing  materials  so  that  the  ICC  and  DEP  could 
connect  recommended  improvements  to  current  products,  CNA  examined  existing 
test  documentation  and  analysis  methodologies.  We  determined  which  content 
could  be  preserved  and  which  content  should  be  eliminated  in  order  to  establish 
traceability  between  ICC  test  requirements  and  test  objectives.  CNA  reviewed  the 
ICC  test  planning  guides  that  are  created  by  the  ICC  Test  Management  and  passed 
to  DEP.  The  ICC  Test  Planning  Guide  highlights  the  test  configurations  and  setups 
that  would  allow  the  ICC  to  assess  interoperability  with  a  new  combat  system 
baseline.  Other  documents  reviewed  during  the  DEP  BUR  were  the  Test 
Procedures  created  by  TP&E,  the  Data  Management  and  Analysis  Plans  (DMAPs) 
to  understand  analysis  methodologies,  various  scheduling  documents,  and  finally 
the  Test  Control  Board  (TCB)  Charter  [4],  The  TCB  charter  is  the  backbone  of  the 
entire  interoperability  assessment  process,  thus  it  was  important  to  ensure  that  the 
proposed  Future  State  aligns  with  the  TCB  process  of  managing  DEP  scheduling 
and  programmatic  reviews.  The  TCB  is  composed  of  members  from  both  DEP  and 
ICC.  It  meets  periodically  throughout  the  year  to  determine  which  ICC-requested 
tests  the  DEP  will  be  able  to  accommodate. 
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Observed  limiting  factors  to  PEP  testing  process 

•  Interoperability  assessment  testing  process  faces  challenges 

Limited  communication 
Segmented  steps 

What  are  we  really  testing  and  why? 

•  Distributed  sites  where  staff  have  limited  interaction 

•  Vertical  communication  pathways 

•  Narrow  definitions  of  tasks  and  roles 

•  ICC  and  DEP  operate  independently 

•  Strategies  to  avoid  above  conditions 

Cross  functional  teams 

Collaborative  products  that  go  directly  to  management 


CNA 


While  conducting  the  BUR,  CNA  collected  data  and  observations  that  highlighted 
needed  areas  of  improvement  for  both  the  ICC  and  DEP  functional  areas.  First,  the 
interface  between  DEP  and  the  ICC  is  disconnected  and  creates  limited 
communication  between  the  two  entities.  This  disconnect  drives  many  fundamental 
misunderstandings  when  it  comes  to  both  test  execution  and  analysis.  The  goal  of 
the  DEP  BUR  is  to  help  address  these  misunderstandings  by  establishing 
traceability  from  the  ICC  Test  Planning  through  the  DEP  testing  and  analysis. 

Next,  limited  cross  functional  communications  within  the  DEP  functional  areas  occur 
partly  due  to  the  geographically  distributed  nature  of  the  organization  where  staff 
may  have  limited  interactions.  CNA  reviewed  some  strategies  to  help  avoid  these 
communication  pitfalls  such  as  establishing  cross  functional  teams  and  generating 
collaborative  products  across  all  necessary  functional  areas  within  both  the  DEP 
and  ICC  that  are  passed  directly  to  the  program  management.  These  cross 
functional  teams  include  participation  by  both  the  DEP  and  ICC  and  allow  all 
functional  areas  in  both  entities  to  be  more  invested  as  stakeholders  in  the 
interoperability  assessment  testing  process.  This  suggested  process  change 
introduces  a  moderate  paradigm  shift  from  the  current  interoperability  assessment 
testing  methodology  should  help  eliminate  many  of  the  communication  gaps  across 
functional  areas  and  also  to  higher  program  management. 

Finally,  the  ICC  and  DEP  functional  areas  cannot  continue  to  operate  as 
independent  organizations.  For  both  the  ICC  and  DEP  programs  to  remain 


13 


(continued) 

relevant  to  US  Naval  Fleet  Forces,  collaborations  must  be  established  and 
maintained  especially  in  situations  where  resources  may  be  constrained  but  the 
complexity  and  duration  of  future  testing  events  increases  (i.e.,  DEP  is  required  to 
do  more  with  less). 
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Reality  of  the  BUR 

•  People 

How  can  we  increase  interactions  between  stakeholders? 

What  roles  and  responsibilities  do  people  have  in  the  new  process? 

•  Process 

How  can  ICC  and  DEP  improve  collaboration  during  interoperability 
assessment  test  cycles? 

•  Products 

What  tools  are  necessary  to  complete  the  process? 

What  information  must  be  passed  between  participants  to  support 
testing  and  assessment? 


CNA 


For  the  DEP  BUR  to  be  successful,  CNA  had  to  address  several  communication 
barriers  and  process  improvements  that  fall  into  three  categories:  people,  process, 
and  products  that  are  included  in  each  DEP  functional  area  along  with  the  ICC. 
Many  of  the  communication  barriers  and  impediments  to  a  successful 
interoperability  assessment  testing  process  can  be  addressed  by  dissection  of  the 
RIE  proposed  Future  State  process. 

People  -  The  proposed  Future  State  process  created  from  the  RIE  events  requires 
involvement  by  all  DEP  functional  areas  earlier  in  the  ICC  planning  phases,  along 
with  requiring  additional  ICC  oversight  during  DEP  test  planning,  analysis,  and 
reporting  to  the  program  management.  During  the  ICC  RIE  Future  State 
development,  there  was  insufficient  time  allowed  for  the  specific  roles  and 
responsibilities  to  be  defined.  As  a  result,  CNA  has  collaborated  with  the  ICC  and 
DEP  functional  leads  to  make  recommendations  on  these  roles  and  responsibilities 
in  the  Future  State. 

Process  -  The  Future  State  process  improves  the  limited  communications  between 
the  DEP  functional  areas  and  the  ICC  by  increasing  their  interactions  during  the 
entire  interoperability  assessment  testing  process.  These  increased  interactions  in 
the  Future  State  process  should  allow  for  better  information  exchange  between  all 
stakeholders. 

Products  -  CNA  examined  possible  improvements  to  existing  tools  that  would 
allow  traceability  from  requirements  to  test  objectives  to  be  established. 
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Methodology  

1 .  Define  Roles  &  Responsibilities  based  on  the  proposed  ICC  &  DEP 
interoperability  assessment  process 

■  Defines  who  is  responsible  and  accountable 

2.  Establish  timeline  for  multiple  DEP  test  events  in  FY09 

•  Highlights  new  planning  considerations  and  modified  workload 

3  Create  traceability  within  the  ICC/DEP  testing  process 

■  High-level  ICC  requirements  ->  DEP  Test  Objectives 

•  Original  task 

4.  Revise  targeted  analysis 

■  High-level  ICC  requirements  ->  DEP  Test  Objectives  ->  DEP  Analysis 

■  Analysis  priorities  can  be  established  from  the  requirements  trace 

5.  Refocus  ICC  Handoff  Briefing  Product  (CAB  D0018373.A2/Final) 

•  High-level  ICC  requirements  ->  DEP  Test  Objectives  ->  DEP  Analysis  ->  ICC 
Assessment 

-  Connects  results  to  High-level  ICC  requirements 

CNA 


CNA,  DEP,  and  ICC  have  worked  to  develop  several  products  and 
recommendations  for  improvements  to  interoperability  assessment  testing  process 
while  completing  the  BUR.  In  particular,  we  examined  those  areas  that  require 
increased  collaboration  between  the  DEP  and  the  ICC. 

First,  CNA  collaborated  with  ICC  members  and  DEP  functional  leads  to  establish 
the  roles  and  responsibilities  (R&R)  for  the  RIE  Future  State  process.  The  R&R 
recommendations  were  necessary  because  time  constraints  during  the  DM&A/ICC 
RIE  execution  did  not  permit  the  original  completion  of  this  task.  This  is  a  critical 
step  in  the  implementation  of  a  recommended  improved  process.  As  part  of  the 
BUR,  these  R&R  recommendations  were  proposed  for  every  process  step  in  the 
ICC/DM&A  Future  State.  The  responsibility  and  accountability  charts  will  be 
examined  later  in  more  detail. 

Then,  the  DEP  PM  requested  that  CNA  demonstrate  the  effects  of  applying  the 
Future  State  to  one  entire  FY.  In  response,  we  created  a  detailed  timeline  using  all 
existing  and  proposed  DEP  and  ICC  process  recommendations.  This  timeline  could 
help  assist  in  setting  milestones  and  deliverables  for  upcoming  events.  The  timeline 
makes  visible  overlaps  in  workloads  across  tests  that  are  not  obvious  when 
considering  a  single  event. 

The  most  critical  connection  that  needs  to  be  made  is  in  the  handover  of  test 
planning  materials  from  the  ICC  to  all  DEP  functional  areas.  The  main  improvement 
at  this  ICC/DEP  interface  is  creating  a  high  level  ICC  master  requirements  trace  that 
addresses  the  original  scope  of  the  BUR.  The  requirements  trace  will  demonstrate 
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(continued) 

traceability  from  the  ICC  requirements  to  the  DEP  test  objectives  and  will  allow  all 
stakeholders  from  the  ICC  and  the  DEP  to  better  understand  the  requested  ICC  test 
cases  and  test  objectives.  CNA  created  a  traceability  matrix  that  maps  ICC 
requirements  to  an  existing  set  of  interoperability  assessment  test  objectives. 

Also,  we  propose  a  revision  of  the  DM&A  targeted  analysis  methodology.  The 
targeted  analysis  builds  upon  the  requirements  trace  matrix  and  connects  purposed 
targeted  analytical  techniques  to  both  the  test  objectives  and  the  underlying  high- 
level  ICC  requirements.  The  targeted  analysis  can  allow  DM&A  to  conduct  a  more 
focused  analysis  around  the  ICC  test  objectives  and  will  establish  analysis  priorities 
that  better  fit  the  ICC  needs. 

Finally,  a  revised,  simplified  handoff  briefing  that  can  be  used  as  a  template  for 
delivering  test  results  to  the  DEP  PM  and  the  ICC  has  been  developed  [5].  Some  of 
the  suggestions  for  this  revised  handoff  brief  involve  elements  that  demonstrate 
traceability  from  analytic  results  back  to  ICC  requirements.  The  interoperability 
issues  that  are  reported  out  to  the  DEP  PM  and  ICC  in  the  handoff  brief  should 
show  a  direct  connection  to  the  initial  ICC  high-level  requirements.  The  brief  is  also 
restructured  for  simplification  in  order  to  accurately  deliver  important  interoperability 
issues  discovered  within  the  DEP  testing  to  the  ICC  and  DEP  Program 
Management. 
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Much  of  the  BUR  analysis  focuses  heavily  on  the  beginning  of  the  testing  process 
during  the  ICC  planning  stages.  As  we  stated  on  the  data  sources  slide  earlier,  a 
product  of  the  DM&A  and  ICC  RIE  was  an  improved  Future  State  process  map.  The 
Future  State  process  diagram  shown  on  the  slide  above,  was  produced 
collaboratively  by  the  ICC  and  DEP.  We  have  generated  a  larger,  more  readable 
version  of  the  Future  State  in  Appendix  G. 

The  entire  Future  State  can  be  divided  into  two  distinct  phases  with  the  ICC  and 
DEP  trading  off  on  the  primary  role  responsibility  as  highlighted  in  the  slide  above. 
The  ICC  planning  phase  is  the  portion  of  the  process  diagram  that  is  on  the  left  side 
of  the  handover  division  line.  This  line  represents  where  the  primary  responsibility 
transitions  from  the  ICC  to  the  DEP  in  the  overall  interoperability  assessment  test 
process.  In  the  earliest  portion  of  the  Future  State  timeline,  the  ICC  has  the  primary 
responsibility  for  selecting  those  high-level  warfare  requirements  that  are  suitable  to 
be  tested  in  the  DEP. 

Historically,  the  ICC  test  management  has  not  required  participation  by  the  DEP 
functional  leads  in  the  early  planning  phases.  Now,  with  the  Future  State,  the  DEP 
functional  leads  are  established  as  stakeholders  early  in  the  ICC  test  planning 
process  as  a  check  and  balance  to  ensure  that  proposed  requirements  can  be 
tested  and  sufficient  data  can  be  collected  within  the  DEP. 
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(continued) 

The  Future  State  diagram  incorporates  a  significant  increase  in  up-front 
participation  by  the  DEP  functional  leads.  The  ICC  test  planning  time  has  been 
extended  to  a  17-week  workup  period  that  may  create  extended  overlaps  between 
event  timelines  for  a  given  fiscal  year. 

In  the  Future  State,  the  handover  from  the  ICC  to  the  DEP  occurs  at  the  delivery  of 
a  formal  test  planning  document  that  is  collaboratively  developed  between  the  DEP 
and  the  ICC  stakeholders.  After  the  handover  occurs,  the  DEP  functional  leads 
assume  primary  role  responsibility  and  are  responsible  for  test  execution  and  the 
delivery  of  the  analysis  results  to  the  ICC. 
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Each  box  in  the  diagram  represents  a  specific  process  task.  For  illustration 
purposes  in  this  document  we  have  divided  the  Future  State  process  diagram  into 
three  sections.  The  ICC  planning  phase  is  outlined  in  the  slide  above.  On  later 
slides  we  will  show  both  the  DEP  TP&E  and  DM&A  phases  after  the  ICC  and  DEP 
primary  role  handoff  occurs. 

In  the  ICC  planning  phase,  along  with  the  critical  handover  to  the  DEP  functional 
leads,  there  are  two  key  process  steps  that  set  the  Future  State  apart  from  its 
predecessor:  (1)  defining  key  DEP  and  ICC  stakeholders  for  the  development  of  a 
formalized  test  planning  guide  and  (2)  creating  traceability  between  requirements 
and  test  objectives.  The  key  stakeholders  are  defined  earlier  in  the  process  flow, 
and  this  will  bring  the  DEP  functional  leads  into  the  test  process  much  earlier  than  in 
the  past.  The  overall  testing  process  will  benefit  if  the  DEP  functional  leads  have  a 
better  understanding  of  the  ICC’s  motivations  and  initial  test  requests,  while  the  ICC 
will  be  more  familiar  with  the  DEP  capabilities  to  test  and  analyze  requirements  that 
are  to  be  tested  in  the  DEP.  This  is  especially  important  for  new  combat  systems 
and  their  related  subsystems  that  have  never  been  tested  within  the  DEP. 

The  final  step  before  the  primary  role  handover  from  ICC  to  DEP  is  the  delivery  of  a 
formal  test  planning  document  that  clearly  maps  the  ICC  requirements  to  the  test 
objectives  and  the  planned  analysis. 
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DEP-led  Future  Process/State  (1  of  2) 
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Process  promotes  continued  ICC  &  DEP  collaboration 


CNA 


After  the  ICC  handover  of  the  formal  test  planning  document,  all  test  planning, 
execution,  data  management,  and  analysis  responsibilities  are  passed  to  the  DEP 
functional  areas.  The  DEP  planning  portion  of  the  Future  State  diagram  has  been 
split  into  two  pieces:  the  TP&E  tasks  (above)  and  the  DM&A  tasks  (next  slide).  The 
tasks  in  the  slide  above,  driven  primarily  by  TP&E,  progress  in  parallel  to  the  DM&A 
tasks  on  the  next  slide.  For  example,  related  steps  are  shown  on  both  process 
diagrams  such  as  “Hold  TPWG  #1”.  E&O,  while  always  maintaining  the  overall 
readiness  of  the  DEP,  also  participate  in  several  of  the  specific  steps  outlined  on 
these  two  slides.  As  with  the  proposed  ICC  planning  phase,  now  the  DEP  functional 
areas  must  maintain  collaboration  with  the  ICC. 
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Here  we  present  the  DM&A  Future  State  process  steps  that  lead  to  test  execution. 
These  steps  occur  in  parallel  to  the  steps  on  the  previous  page.  Many  of  these 
steps  highlight  areas  where  contact  with  the  ICC  or  other  DEP  functional  areas 
would  be  necessary.  We  did  not  consider  in  this  BUR  the  steps  labeled  “Part  of  the 
DM&A  Process  Map”  above.  The  DEP  addressed  these  steps  in  more  detail  in  a 
separate  process  improvement  event,  and  they  are  beyond  the  scope  of  the  BUR. 
Test  execution  and  analysis  follow  the  steps  seen  above  and  were  not  considered  in 
detail  during  the  RIE,  although  we  introduce  ideas  for  revising  data  analysis  later  in 
this  report. 
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The  ICC  and  DEP  functional  leads,  in  collaboration  with  CNA,  have  established 
roles  and  responsibilities  for  the  entire  Future  State  testing  process.  We  solicited 
comments  from  all  DEP  functional  leads  and  all  members  of  the  ICC  on  our 
preliminary  assignment  of  roles.  Our  final  recommendations  for  each  process  step, 
which  can  be  found  in  their  entirety  in  Appendices  A  and  B,  incorporate  these 
comments.  For  each  process  box  in  the  proposed  Future  State  diagram,  CNA  has 
recommended  the  person  or  organization  responsible  and  accountable  for 
completion  of  that  process  step.  In  addition,  we  identify  who  the  responsible 
person(s)  or  organization(s)  should  be  collaborating  with  in  order  to  make  that 
process  step  successful.  For  illustration  purposes,  we  provide  a  sample  column  of 
process  steps  in  the  ICC  planning  phase  on  the  next  page. 
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This  slide  describes  a  specific  example  of  the  format  used  to  establish  roles  and 
responsibilities  in  the  ICC-DM&A  Future  State.  We  list  each  step  and  associated 
subtasks  from  the  Future  State.  The  specific  example  above  demonstrates  the 
increased  interaction  during  the  ICC  planning  phase  of  DEP  testing.  Developing 
Test  Cases,  Test  Configurations,  and  Setup  Requirements  are  all  responsibilities  of 
the  ICC,  but  also  each  one  of  these  process  steps  requires  analysis  and  input  from 
the  appropriate  DEP  functional  area. 

On  the  right-hand  side  of  each  slide  is  a  timeline  adapted  from  the  Future  State 
process  map.  We  include  an  arrow  on  the  timeline  to  show  process  progress  for 
each  column.  The  steps  on  a  single  slide  are  not  meant  to  be  taken  as  linear  in 
time;  they  may  occur  simultaneously.  However,  the  process  does  flow  forward  in 
time  from  one  column  to  the  next  in  the  overall  Future  State  diagram.  The  full  set  of 
roles  and  responsibilities  is  included  in  Appendices  A  and  B  to  this  report. 
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Establish  timeline  for  multiple  DEP  test  events  in  FY09 


FY  09  FY  10 

Oct  N  D  J  F  M  A  May  J  J  A  S  Oct  N  D  J  F  M 


Test  Execution 
ICC:  17  wks 


□  TP&E.  8-10  wks 

|  DM&A,  14-15  wks  after  test  execution 
0  E&O 

•  The  new  timeline  increases  collaboration  between  ICC  and  DEP  functional 
areas 

•  Increases  upfront  planning  time 

•  Redistributes  workload 

•  Adheres  to  TCB  Charter 

CNA 


Since  the  DEP  executes  multiple  test  events  within  a  given  fiscal  year,  we  wanted  to 
determine  the  impact  on  workload  when  multiple  events  modeled  after  the  Future 
State  are  overlaid  for  a  given  fiscal  year.  In  the  slide  above,  we  have  developed  a 
general  future  state  timeline  for  multiple  events  executed  within  a  fiscal  year.  On 
this  slide,  we  demonstrate  how  the  new  Future  State  impacts  the  overall  workload 
for  all  ICC  and  DEP  functional  areas  in  a  fiscal  year  when  multiple  events  are 
planned. 

As  we  discussed  on  prior  slides,  the  Future  State  process  requires  a  17-week  lead 
time  for  planning.  The  DEP  program  plans  approximately  3  to  4  ICC  interoperability 
tests  for  a  given  year.  The  notional  timeline  diagram  above  shows  the  planning 
overlap  for  all  participants  in  a  DEP  event.  For  example,  during  event  A,  the  initial 
grey  box  represents  the  ICC  planning  phase  with  collaboration  from  TP&E  (pink) 
and  E&O  (green)  and  DM&A  (blue)  groups.  The  ICC  continues  to  collaborate  with 
the  DEP  functional  areas  when  the  lead  role  switches  from  the  ICC  to  DEP 
functional  areas  in  event  A  (approximately  February  2009).  Of  note  is  that  when  the 
lead  role  switches  for  event  A,  planning  for  event  B  has  already  begun.  This 
succession  of  planning  efforts  will  persist  through  the  evolution  of  the  fiscal  year 
while  the  staff  across  the  DEP  and  ICC  remains  constant.  This  redistribution  and 
stacking  of  workload  requires  refocused  planning  across  the  ICC  and  DEP  in  order 
for  event  planning  progress  to  move  in  a  forward  direction. 
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FY09  PEP  Sched u I e  -T argeted  Events 


CNA 


For  the  upcoming  DEP  events  in  the  2009  fiscal  year,  CNA  has  targeted  the  Cruiser 
Modernization  (CGM)  Interoperability  Assessment  (IOPA)  Test  as  a  dry  run  of  the 
Future  State.  The  CGM  IOPA  is  planned  for  execution  in  May  2009.  On  this  slide, 
we  have  assembled  a  detailed  timeline  on  a  week-by-week  basis  that  includes 
process  steps,  responsible  functional  areas,  and  deliverables  for  the  CGM  IOPA 
overlaid  with  the  last  IOPA  event  of  the  FY  that  is  currently  planned  but  unspecified. 
The  timeline  is  included  full-size  in  Appendix  F.  Because  the  Future  State  was 
heavily  influenced  by  ICC  test  planning,  we  then  added  established  timelines  from 
current  Plan  of  Action  and  Milestones  (POA&M)  published  in  test  procedures  and 
Data  Management  and  Analysis  Plans  (DMAP)  that  largely  remain  unchanged  from 
the  former  IOPA  test  process.  This  combined  timeline  incorporates  the  lessons 
learned  from  the  RIE  event  and  also  is  an  accurate  representation  of  the  analytical 
effort  that  is  necessary  after  test  execution. 

The  reality  of  the  increased  ICC  planning  phase  and  the  early  collaboration  of  the 
DEP  functional  areas  is  that  planning  for  the  first  event  at  the  beginning  of  a  fiscal 
year  must  occur  five  to  six  months  prior  to  the  start  of  the  fiscal  year.  This  planning 
requirement  presents  various  funding  and  manning  challenges  for  the  DEP  PM. 
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Event-based  deliverables  to  DEP  Program  Manager 


•  Formal  Test  Planning  Guide 

ICC,  DM&A,  E&O,  TP&E 

•  Test  Procedures 

TP&E 

•  Test  Readiness  Review 

DM&A,  E&O,  TP&E 

•  Data  Management  and  Analysis  Plan 

DM&A,  TP&E,  ICC  Assessment 

•  ICC  Handoff/NAVSEA  Report 

DM&A 


CNA 


The  previous  sections  of  this  document  focused  on  the  roles  that  the  ICC  and  DEP 
functional  areas  perform  and  the  specific  details  of  the  testing  process.  In  the 
remainder  of  this  document,  we  focus  on  multiple  products,  such  as  the  ICC  Test 
Planning  Guide  and  imbedded  traceability  matrix,  that  should  pass  among  the 
interoperability  assessment  test  stakeholders  as  they  share  information.  We 
believe  that  a  formal  ICC  Test  Planning  Guide  should  be  developed  collaboratively 
between  the  ICC  and  DEP  functional  areas.  The  tools  within  this  guide  will  convey 
requirement  traceability  among  stakeholders  and  between  process  steps. 

Here  we  summarize  the  current  list  of  deliverables  that  correspond  to  event-based 
milestones  on  the  previous  timeline  slide.  All  deliverables  are  carried  over  from  the 
former  interoperability  assessment  process  except  the  formal  test  planning  guide. 
We  also  list  who  should  be  involved  in  producing  each  deliverable.  Previously,  an 
informal  test  planning  guide  had  been  produced  solely  by  ICC  Test  Management 
and  was  delivered  to  the  DEP  at  the  start  of  the  Test  Plan  Working  Group  (TPWG). 
One  of  the  key  recommendations  of  the  ICC/DM&A  RIE  is  for  the  ICC  Test 
Management,  with  the  collaboration  of  all  DEP  functional  areas,  to  produce  a 
formal  test  planning  guide. 
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Requirements  traceability 

•  In  past  events,  Test  Planning  Guide  did  not  directly  reflect  ICC  needs 

-  Formalize  the  connections  between  DEP  testing  and  ICC  requirements 

•  We  used  materials  from  SSDS  10  DEV  event  to  provide  a  model 
requirements  trace. 

•  We  have  modified  existing  tools  for  each  Test  Case 

-  ICC  requirements  trace  matrix 

-  Connects  test  objectives  and  high-level  ICC  requirements 

-  TADIL-CEC  architecture  map 

•  Provides  a  clean,  visual  representation  of  combat  systems  on  each  network 

■  GCCS-M  participants  are  shown  in  a  separate  insert 
Configuration  matrix 

■  Carried  forward  from  previous  ICC  test  planning  materials 

•  Only  lists  suggested  platforms,  not  specific  host  sites 

•  We  recommend  that  each  of  these  tools  be  delivered  in  a  formalized 
Test  Planning  Document  by  the  ICC  Test  Management  prior  to  the  first 
TPWG. 

CNA 


On  this  slide,  we  discuss  the  requirements  trace  that  we  developed  showing  the 
connection  between  ICC  interoperability  requirements  and  DEP  test  objectives.  This 
traceability,  the  core  of  our  original  tasking  from  the  DEP  Program  Manager,  is  the 
foundation  for  improved  test  execution,  data  analysis,  and  interoperability 
assessment  throughout  the  ICC-DEP  testing  process. 

In  order  to  establish  the  traceability  from  the  high-level  ICC  requirements  to  the 
interoperability  assessment  test  objectives,  we  used  materials  from  the  Ship  Self 
Defense  System  (SSDS)  Interoperability  Development  (10  DEV)  08  assessment 
event  to  develop  a  model  requirements  trace  [6,  7].  This  event  was  originally 
planned  to  be  completed  in  FY08  but  has  been  postponed.  For  an  10  DEV  test,  the 
ICC  defines  several  test  cases,  each  representing  a  different  set  of  testing 
conditions.  Each  test  case  is  divided  into  more  specific  test  objectives.  The  ICC’s 
design  of  test  cases  and  test  objectives  is  driven  by  high-level  warfare 
requirements.  However,  the  current  and  informal  ICC  Test  Planning  Guide  does  not 
directly  make  the  connection  between  high-level  ICC  requirements  and  test 
objectives.  We  recommend  that  the  description  of  each  test  objective  in  the  test 
planning  material  includes  the  three  tools  listed  in  the  slide  above:  a  requirements 
trace  that  we  are  establishing,  a  simplified  link  architecture  map,  and  a  combat 
system  configuration  matrix.  The  simplified  link  architecture  map  and  combat 
system  configuration  matrix  are  modifications  of  items  currently  included  in  the 
informal  ICC  Test  Planning  Guide.  We  will  describe  each  of  these  tools  and  provide 
examples  of  their  use.  These  tools  should  be  part  of  the  test  planning 
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(continued) 

documentation.  The  collaborative  ICC/DEP  functional  area  formal  test  planning 
guide  will  be  delivered  by  the  ICC  test  management  lead  to  the  broader  DEP 
community  prior  to  the  first  Test  Planning  Working  Group  (TPWG)  as  outlined  in  the 
above  FY09  target  events  schedule. 
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ICC  requirements  flow 


ICC  Requirements 


High-Level  f  Test 
f  Warfare  I  Requirement 
Functional  \ 


Test  Planning  Guide 


Mission  ^ 
Function*1 


Requirement 


High-Level 

Warfare 

Functional 

Requirement 


Test 
Requirement 

Test 

Requirement 


Test 
Requirement 


X 


Test  Objective^ 

Test  Objective 
Test  Objective 

Test  Objective 
Test  Objective-^ 


Analysis 


>Test  Case 


Targeted  analysis 
Targeted  analysis 

Targeted  analysis 


General 


Specific  General 
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The  ICC  uses  a  set  of  high-level  requirements  to  guide  the  composition  and 
configuration  of  a  strike  force  participating  in  an  interoperability  assessment  test. 
The  requirements  are  derived  from  various  sources,  including  military  standards 
(e.g.,  MIL-STD  6016C  [8]),  Navy  doctrine,  and  Joint  publications.  We  do  not  discuss 
the  original  sources  of  requirements  here  because  they  are  beyond  the  current 
scope  of  the  BUR. 

These  requirements  start  with  very  general  descriptions  of  Mission  Functions.  Each 
Mission  Function  is  composed  of  several  more  specific  Warfare  Function 
Requirements.  A  list  of  the  requirements  relevant  to  the  DEP  appears  in  Appendix 
D.  Very  recently,  the  ICC  has  further  divided  each  Warfare  Functional  Requirement 
into  several  detailed  Test  Requirements.  As  an  example,  one  Mission  Function  is 
Surveillance  Track  Reporting,  which  is  divided  into  six  High-Level  Warfare 
Functional  requirements,  each  focusing  on  a  different  capability. 

In  the  current  testing  process,  the  ICC  test  management  includes  their  required  test 
cases  and  associated  test  objectives  within  their  informal  test  planning  guide. 
Nominally,  this  includes  required  combat  systems  and  the  desired  tactical 
configurations  for  each  test  case.  Each  test  case  contains  several  test  objectives.  In 
the  past,  there  has  been  no  clear  connection  between  the  requirements  and  the 
objectives.  This  disconnect  has  resulted  in  incomplete  test  execution  and 
incomplete  analysis  results.  Therefore,  the  requirements  traceability  must  pass  not 
only  into  the  test  objectives  but  also  into  all  future  analysis  methodologies. 
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How  we  created  the  traceability  matrix  

(1)  Determine  which  ICC  requirements  are  DEP  Specific  -  already  completed  by  the  ICC 
Requirements  Lead 


DEP  SPECIF  1C  ICC  Maatar  Raqutrmanta 

Function 

Subfunction 

Functional  Roqulramant 

Mutual  Tracking 
MT-DEP 

SlAP 

MT11  -  Evaluata  capability  of  platform  whan 
operating  with  lha  Slnka  Force  to  maintain  a 
single  track  per  object  (SlAP  Clarity) 

MT12  •  Evatuata  capability  of  platform  whan 
operating  with  the  Stnka  Force  to  maintain  a 
continuous  LTN  and  CEPN  (SlAP  Continuity) 

MT 1 3  Evaluata  capability  of  platform  when 
operating  with  tha  Stnka  Force  to  maintain  a 
common  picture  such  that  tha  tracks  held  by 
each  participant  have  the  same  LTN  LTN/CEPN 
pairing  tD  position,  on  tha  same  object 
(SlAP  Commonality). 

(2)  Map  each  test  case  and  test  objective  directly  (D)  or  indirectly  (I)  to  functional  requirements 


Test  Casa 

Object  tvas 

Mutual  tracking 

SlAP 

MT11 

MT12 

MT13 

Test  Casa  1; 

Teat  Strike  Group 

Compare  SlAP  results  to  RSG 

D 

D 

D 

Verify  the  SSDS  is  able  to  operate  with  High  TN  lU  units 
and  other  interoperability  fixes  based  on  SSDS  and 

CDLMS  VDDs  or  fix  lists 

D 

D 

D 

Examine  the  ability  of  the  SG  to  track  and  report  the 

AAW  situational  awareness  and  CTP  for  the  SG's  AOR 
to  the  COP  via  GCCS-M/CST  and  LINK  networks 

1 

1 

1 

Direct  (D)  data  collected  to  support  a  specific  test  case/objective  can  be  used  directly  to  address 
whether  a  requirement  has  heen  met 


Indirect  (I)  data  collected  to  support  a  specific  test  case/objective  when  comhined  with  other  data 
^  from  a  DEP  test  or  other  sources  may  be  used  to  address  whether  a  requirement  has  been  met 

CNA 


We  accomplished  the  requirements  trace  in  two  steps.  First,  we  took  the  master  list 
of  ICC  requirements  and  determined  which  requirements  could  be  tested  within  the 
DEP  environment,  already  denoted  by  the  ICC  requirements  lead  as  being  suitable. 
The  first  table  above  shows  a  subset  of  the  DEP-relevant,  high-level  ICC 
requirements,  which  can  be  found  in  their  entirety  in  Appendix  D.  Second,  based  on 
material  from  the  SSDS  10  DEV  event  [6],  we  thoroughly  examined  each  test  case 
and  corresponding  test  objectives  and  assigned  appropriate  ICC  Functional 
Requirements.  The  requirements  may  map  either  directly  or  indirectly.  If  a 
requirement  maps  “directly”  to  a  test  objective,  it  indicates  that  data  measured  in  a 
DEP  test  apply  directly  to  the  requirement  to  assess  whether  the  requirement  has 
been  satisfied.  An  “indirect”  map  implies  that  DEP  data  can  support  an  ICC 
requirement,  but  do  not  specifically  satisfy  the  objective;  additional  data  from  other 
tests  may  be  required.  The  resulting  matrix  highlights  the  rationale  for  each 
objective  tested,  based  on  official  ICC  requirements.  We  expand  the  above  matrix 
on  the  next  page  to  provide  a  more  detailed  example. 
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ICC  requirements  trace  matrix 


Test  Case 

Objectives 

Mutual  Tracking 

Correlation 

TN  MGMT 

SIAP 

Test  Case  1: 

Test  Strike 
Group 

1.  Compare  SIAP  results  to  RSG. 

MT5 

MT6 

MT7 

MT10 

MT11 

MT12 

MT13 

D 

D 

D 

2  Verify  the  SSDS  is  able  to  operate  with  High  TN  IU  units  and 
other  interoperability  fixes  based  on  SSDS  andCDLMS  VDDs  or 
fix  lists. 

D 

D 

D  D  D 

D 

D 

3  Examine  the  ability  of  the  SG  to  track  and  report  the  AAW 
situational  awareness  and  CTP  for  the  SG's  AOR  to  the  COP  via 
GCCS-M/CST  and  Link  networks 

' 

' 

1 

Direct  (D)  -  data  collected  to  support  a  specific  test  case/objective  can  be  used  directly  to  address  whether  a  requirement 
has  been  met 


Currently  delivered  by  ICC  to  DEP  in  Test  Planning  Guide  Proposed  trace  component 


Indirect  (I)  -  data  collected  to  support  a  specific  test  case/objective  when  combined  with  other  data  from  a  DEP  test  or 
other  sources,  may  be  used  to  address  whether  a  requirement  has  been  met 

•  Implied  connections  to  ICC  requirements  become  defined 

CNA 


Currently,  the  ICC  delivers  its  test  cases  and  test  objectives  to  the  DEP  in  a  table 
similar  to  the  first  two  columns  above  (taken  from  the  SSDS  10  DEV  08  test  [6]). 
However,  there  is  no  visible  connection  to  the  underlying  ICC  requirements  that 
drive  these  test  cases  and  objectives. 

We  propose  adding  a  set  of  columns  to  the  test  case/test  objective  table  (as  shown 
above  on  the  right-hand  side  of  the  table)  that  clearly  demonstrates  how  each 
requirement  maps  to  a  specific  test  objective.  The  new  columns  will  contain  all  of 
the  ICC  Functions,  Subfunctions,  and  Functional  Requirements  maximizing  the 
amount  of  information  communicated  from  the  ICC  to  all  DEP  functional  areas 
about  the  motivation  behind  each  test  case  and  composite  test  objective. 

In  each  column,  the  ICC  can  indicate  which  test  objectives  are  based  on  each 
requirement.  Again,  objectives  may  map  directly  or  indirectly  to  each  requirement. 
The  table  above  is  a  portion  of  the  complete  requirements  trace.  Here  we  take  Test 
Case  1 ,  the  Test  Strike  Group,  and  show  a  sample  of  three  of  its  test  objectives. 
The  columns  to  the  right  contain  the  ICC  requirements.  In  this  example,  we  show 
Mutual  Tracking  as  one  of  the  ICC  Mission  Functions.  Correlation,  Track  Number 
(TN)  Management,  and  Single  Integrated  Air  Picture  (SIAP)  are  subfunctions  of  the 
Mission  Function.  Each  subfunction  is  further  divided  into  specific  High-Level 
Warfare  Functional  Requirements,  abbreviated  here  as  MT5,  MT6,  etc.  From  this 
trace  matrix,  we  see  that  Test  Objective  1  of  Test  Case  1  is  designed  to  directly 
addresses  the  SIAP  requirements. 
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(continued) 

Test  Objective  2  is  based  on  all  of  the  Mutual  Tracking  requirements,  and  DEP 
testing  can  directly  satisfy  each  of  those  requirements  for  this  objective.  In  addition, 
SIAP  Mutual  Tracking  applies  to  Test  Objective  3,  but  DEP  testing  will  only 
indirectly  satisfy  this  Objective.  The  full  trace  for  Test  Case  1  and  all  other  SSDS  10 
DEV  08  test  cases  is  in  Appendix  C. 

As  we  discussed  earlier,  the  goal  of  creating  the  completed  requirements  trace  was 
to  eliminate  the  implied  connections  between  requirements  and  objectives  and 
define  these  components  explicitly  to  participants  including  DEP  leadership, 
analysts,  test  planners,  site  engineers,  and  operators.  It  is  also  important  that  while 
ICC  Test  Management  is  responsible  for  delivering  and  maintaining  this  traceability 
matrix,  DEP  functional  leads  and  other  necessary  ICC  members  should  also  be 
consulted  as  defined  in  the  ICC  Future  State. 


33 


Utilities  of  Requirements  Traceability 

•  ICC  goals  and  motivation  are  apparent  to  all  stakeholders 

•  ICC  can  review  test  cases  to  identify  potential  testing  redundancies 
and  gaps 

•  TP&E  &  DM&A  may  establish  testing/analysis  priorities  based  on 
ICC  requirements 

•  DM&A  may  use  established  traceability  to  organize  and  deliver  final 
results  to  the  ICC 

CNA 


Explicitly  mapping  the  underlying  ICC  requirements  to  test  objectives  provides 
critical  connections  between  the  requirements  the  ICC  is  hoping  to  assess  and  the 
planned  test  objectives.  Several  functional  areas  within  the  ICC  and  DEP  will  benefit 
from  this  trace. 

First,  the  requirements  trace  will  allow  the  ICC  to  determine  if  current  test  cases  and 
test  objectives  over-test  or  under-test  each  of  the  requirements  for  a  given  event. 

For  example,  the  requirements  dealing  with  SIAP  attributes  appear  in  multiple  test 
Cases  and  objectives;  it  may  be  possible  to  restructure,  or  consolidate  these 
objectives  if  there  are  potential  redundancies. 

Because  of  the  requirements  trace,  the  DEP  and  ICC  functional  leads  that  are 
responsible  for  planning  and  executing  the  test  will  have  a  better  understanding  of 
the  requirements  being  tested  in  each  test  case.  On  occasion,  due  to  technical 
difficulties,  a  test  case  cannot  be  completed  as  planned.  The  requirements  that 
drive  each  test  case  can  help  all  stakeholders  decide  which  systems  can  be 
substituted  and  which  ones  are  absolutely  necessary  for  test  execution  based  on 
the  requirements  that  are  being  addressed.  Also,  due  to  time  limitations,  sometimes 
it  is  not  possible  to  complete  all  test  cases  that  are  planned  for  a  given  event.  When 
this  occurs,  the  requirements  trace  can  aid  the  test  director  and  ICC  Test 
Management  in  prioritization  of  test  cases;  the  test  director  can  then  focus  on  those 
test  cases  that  address  the  most  important  requirements  for  assessment. 
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Interoperability  Assessment  Test  Configuration  Table 

TADIL-CEC  architecture 


CNA 


We  believe  two  additional  tools  can  help  support  traceability  of  ICC  requirements. 
These  tools  should  also  be  part  of  the  formal  ICC  test  planning  guide  and  should  be 
carried  forward  through  succeeding  test  documentation  that  results  from  the 
planning  guide  (e.g.,  Test  Procedures,  DMAP).  The  next  tool  we  examined  is  the 
network  diagram  that  shows  the  tactical  data  information  link  (TADIL)  and 
Cooperative  Engagement  Capability  (CEC)  architecture.  In  the  current  test  planning 
documentation,  the  TADIL-CEC  architecture  diagram  provides  a  graphical 
representation  of  the  link  connectivity  of  the  participating  units  in  each  test  case  [6], 
This  diagram  is  frequently  referred  to  as  the  OV-1  diagram,  for  Operational  View-1 
from  Department  of  Defense  Architecture  Framework  (DODAF)  terminology.  An 
OV-1  is  a  high-level  operational  concept  graphic.  One  of  its  intentions  is  to  stress 
and  describe  system  connectivity.  In  the  OV-1  shown  in  the  slide  above,  Link-11, 
Link-16,  and  CEC  connectivity  are  each  drawn  in  different  colors. 

The  OV-1,  while  useful,  has  shortcomings  when  used  as  a  technical  diagram  for 
DEP  test  planning  and  execution.  In  these  cases,  it  can  be  difficult  to  follow  all  of 
the  lines  belonging  to  each  network.  In  the  above  example,  the  CEC  network  is 
shown  as  a  circle  above  the  participating  combat  systems.  There  are  three  lines 
connecting  three  combat  systems  through  the  circle.  The  Link-16  connections,  on 
the  other  hand,  are  drawn  directly  between  combat  systems.  For  Link-16,  three 
lines  connect  four  platforms.  These  inconsistencies  may  create  confusion  for  an 
operator  at  one  of  the  testing  sites.  In  other  cases,  all  of  the  networks  will  use  the 
lines  connected  through  a  circle  above  the  platforms. 
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(continued) 

An  additional  challenge  to  using  the  OV-1  diagram  as  a  technical  guide  for 
connecting  combat  systems  during  a  test  is  that  it  does  not  reproduce  clearly  in 
black  and  white.  Because  of  cost  and  speed,  most  documents  are  printed  in  black 
and  white.  When  test  operators  or  analysts  use  this  diagram  without  color,  it  is 
difficult  to  determine  the  definitive  connectivity.  This  confusion  can  lead  to  setup 
errors  in  the  test. 


36 


Configuration  Diagram 


Revised  TADIL-CEC  architecture 


CNA 


-  Link- 16 

.  -  Link-11 

•  CEC 

o 

Data  Forwarder 

We  have  revised  the  OV-1  diagram  in  such  a  way  that  it  is  more  useful  to  all 
functional  areas.  We  suggest  using  the  above  format  as  a  more  technical 
description  of  test  cases.  The  lines  representing  each  tactical  network  are  different 
colors  for  when  printed  in  color,  but  the  different  line  styles  reproduce  the 
architecture  diagram  clearly  in  black  and  white.  The  lines  connect  directly  to  the 
platforms,  making  it  still  easier  to  read.  This  particular  test  case  called  for  the 
inclusion  of  the  Global  Command  Control  System-Maritime  (GCCS-M);  we  include 
the  architecture  for  this  network  as  an  inset.  Other  features  of  the  diagram  include 
different  shapes  for  the  different  roles  a  combat  system  plays  within  a  strike  group. 
For  example,  a  diamond  designates  the  data  forwarder  unit  between  Link-16  and 
Link-1 1 .  In  additional  test  cases  in  Appendix  C,  certain  combat  systems  are  set  to 
the  “data  silent”  mode.  We  designate  this  mode  by  darkening  the  circle  for  the  unit. 
The  clarity  of  this  configuration  diagram  is  retained  also  when  the  document  is 
printed  in  black  and  white. 

The  clean  and  simple  approach  taken  here  has  already  proven  its  effectiveness. 
When  we  presented  these  sample  test  case  diagrams  to  the  ICC  based  on 
materials  from  the  SSDS  10  DEV  08-01  test,  ICC  Assessment  identified  two 
mistakes  in  the  test  configuration  that  had  previously  gone  undetected.  We  attribute 
this  to  confusion  in  the  setup  of  the  tactical  configuration. 

We  recommend  this  diagram  replace  or  supplement  the  current  OV-1  diagram 
within  the  document.  The  diagram  above  is  more  technical  and  provides  details  and 
clarity  that  are  essential  for  proper  test  execution. 
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(continued) 

After  the  test  is  complete,  TP&E  should  update  the  diagrams  with  any  configuration 
changes  that  occurred  during  execution.  Due  to  unavoidable  circumstances, 
sometimes  the  test  setup  is  changed  or  modified.  When  this  happens,  it  is  important 
to  make  the  changes  clear  to  all  participants.  An  update  of  the  above  diagram  and 
corresponding  configuration  table  should  be  produced.  The  Test  Director’s  post¬ 
event  execution  summary  report  would  be  an  appropriate  document  to  host  these 
changes  to  be  distributed  to  all  DEP  and  ICC  stakeholders. 
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Configuration  Matrix  

Configuration  Matrix 

•  Carried  forward  from  previous  ICC  test  planning  materials 

•  Only  lists  suggested  platforms,  not  specific  host  sites  at  this  phase  of 
planning 


A  =  Active  D  =  Disabled  D/O  =  Default  On  E  =  Enabled  N/A  =  Not  Applicable 


CNA 


The  final  tool  we  discuss  as  a  part  of  the  test  planning  guide  is  the  test  configuration 
matrix.  We  included  the  configuration  matrix  that  is  carried  forward  from  the  current 
test  documentation  [6].  The  main  difference  is  that  this  table  does  not  include 
specific  software  builds.  Rather,  it  lists  the  general  class  of  combat  system  to  be 
used.  The  ICC  and  DEP  functional  leads  would  determine  specific  software  at  a 
later  time  (at  the  Test  Planning  Work  Group)  based  on  both  testing  needs  and  site 
availability.  This  matrix  contains  necessary  site  setup  information  about  platforms; 
dialed  track  quality  (DTQ);  link  status  for  Link-1 1 ,  Link-16,  Satellite,  CEC,  data 
forwarding,  Link-1 1  track  number  associations,  and  transmitting  pending  filters;  and 
correlation  based  on  gridlock  reference  unit  and  battle  force  correlation 
coordination.  The  architecture  map  on  the  previous  slides  should  be  easily 
produced  from  this  table. 
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Revised  targeted  analysis  

•  CNA  is  in  the  process  of  modifying  the  current  test  objective  targeted 
analysis 

•  CNA’s  intent  of  revising  the  targeted  analysis  is  to  align  it  with  the 
outcome  of  the  BUR 

-  Targeted  analysis  will  be  linked  to  the  test  objectives  and  originating 
requirements. 

Test  objective  analysis  motivation  is  clear  and  unambiguous  to  the  ACB  lead 
and  supporting  analysts. 

•  The  analyst  is  provided  with  a  guided  question  per  test  objective 

•  Cites  available  data  source(s),  calculations,  and  scenario  considerations 
to  conduct  test  objective  analysis  for  an  interoperability  assessment  test 
event. 

•  CNA  will  deliver  a  draft  of  the  targeted  analysis  in  the  beginning  of  FY09 
(CNA  CAB  D0018848.A1). 


CNA 


After  we  established  the  traceability  between  the  ICC  requirements  and  the 
interoperability  assessment  test  objectives  (TO),  we  wanted  to  then  show  the  next 
step  in  the  progression,  revising  the  targeted  analysis  so  it  too  is  in  alignment  with 
the  outcome  of  the  BUR.  CNA  is  developing  a  draft  version  of  revised  targeted 
analysis  for  the  upcoming  SSDS  Interoperability  Assessment  test  executed  in 
September  2008.  We  will  work  with  DM&A  to  ensure  the  targeted  analysis  is 
comprehensive  and  realistic  for  the  current  suite  of  TOs  and  is  adaptable  to  future 
interoperability  assessment  tests. 

The  main  goal  of  revising  the  targeted  analysis  is  to  create  a  critical  link  between 
the  analysis  and  the  originating  ICC  requirements  for  each  TO.  This  linkage 
provides  the  Analysis  Control  Board  (ACB)  lead  and  supporting  analysts  with  a  clear 
motivation  for  each  TO  that  will  aid  them  in  formulating  the  most  complete  and 
proper  piece  of  analysis.  Additionally,  we  provide  the  analyst  with  guided  questions 
per  TO  that  engages  the  analyst,  leading  them  to  consider  the  scope  and  necessary 
steps  of  the  TO  assessment  [9]. 

Our  proposed  revision  of  the  targeted  analysis  includes  the  layout  of  the  critical  tool 
sets  an  analyst  will  need  for  TO  assessment,  including  required  data  sources, 
calculations,  and  specifics  about  how  to  best  utilize  the  available  scenario  to 
discover  interoperability  issues.  These  details  are  a  starting  point  for  the  analyst, 
not  necessarily  the  entire  scope  of  the  potential  analysis.  The  analysts  will  still  need 
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(continued) 

to  isolate  problems  and  determine  the  root  cause  of  each  issue  through  fault- 
isolation  analysis  as  in  past  interoperability  assessment  test  analysis. 

The  revised  targeted  analysis  will  be  delivered  as  a  separate  document  shortly  after 
the  start  of  FY09,  but  we  discuss  the  concept  here  because  it  is  an  integral  product 
based  on  the  outcome  of  the  BUR  [9], 
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Revised  analytical  brief  to  ICC 


•  Maps  results  to  high-level  ICC  Requirements  (denoted  as  “ICC  Function”  in  table) 

•  Provides  a  starting  point  and  prioritization  for  the  ICC  to  perform  assessments 


TOR 

Element 

Issue  Title 

Status 

Severity 

ICC 

function 

Outstanding  TR 

7212-071-005 

C2P 

C2P  dor  a  not  correctly  tranalata  waapon 
typ«  to  Link  11 

TR 

4B 

FC/Engagama 

nt 

7212-071-007 

ACDS 

ACDS  does  not  ctaar  angagament  on  TN 
aftar  braak  angaga  from  ramota  IU 

TR 

4B 

Survalllanca 

7212-071-011 

SSDS 

Platform  axpariancad  CEC  Intarfaca  LTN 
ISMM 

TR 

4B 

Survalllanca 

7212-071-302 

AWS  5  3  9 

AWS  5.3  9  falls  to  claar  mutual  bit  whan 
assuming  R2 

TR 

2C 

Mutual 

tracking 

7212-071-903 

AWS  6.1.7 

AWS  6.1  7  transmits  aurvalllanca 
maaaaga  on  activa  non  C2  JU  aftar 
correlation 

TR 

4B 

Survalllanca 

7212-071-906 

E-17A 

AW  ACS  pairs  aama  L11  track  numbar  to 
multiple  LI 6  track  numbar 

TR.  Documantad 
as  CAP /’Ll  M 

4B 

Survalllanca 

7213-071-005 

CDLMS 

CDLMS  transmits  tarmlnata  control 
maaaaga  with  rafsrenca  TN’  0000 

TR 

2C 

AC 

7213-071-009 

AWS  6.1.7 

AWS  6  17  reports  TN  under  control  by 
wrong  CU 

TR 

2C 

AC 

7213-071-902 

E-2C 

E-2C  Downgredaa  Symbology  to  Local 
Track 

TR 

4A 

Mutual 

tracking 

7213-071-903 

SGS/AC 

Incorrect  SGS/AC  buffars  causa  Incorrect 
ID  changaa  on  Link  16 

TR 

ID 
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Previously,  we  revised  the  CVN  68  Open  Architecture  10  DEV  test  brief  [5].  The 
main  goals  of  the  revised  brief  were  to  streamline  the  results  and  present  them  at  a 
high-level  understandable  to  program  management,  rather  than  to  subject  matter 
experts  who  conducted  the  analysis.  Also,  this  brief  needed  to  communicate  to  the 
ICC  how  the  issues  uncovered  during  testing  mapped  back  to  the  functional 
requirements.  This  revised  brief  product  was  not  part  of  the  original  scope  of  the 
BUR.  However,  we  make  mention  of  this  revised  brief  because  it  demonstrates  how 
issues  found  during  analysis  must  be  referenced  to  feed  the  ICC  assessment 
process. 

On  this  slide,  we  show  a  list  of  high  priority/severity  issues  from  the  CVN  68  10  DEV 
test  and  their  corresponding  functional  requirement  [5].  Each  line  of  the  table 
describes  a  different  problem  found  by  the  analysis  team.  The  TOR  number  is  the 
Test  Observation  Report  number  for  the  observed  issue.  The  element  is  the  combat 
system  or  subsystem  responsible  for  the  issue.  The  status  column  indicates 
whether  the  issue  is  formally  accepted  as  a  Trouble  Report  (TR)  by  the  combat 
system  developer.  The  severity  column  assigns  a  frequency  (A-E,  A  the  highest) 
and  severity  (1-5,  1  the  highest)  to  the  issue.  The  final  column,  ICC  Function, 
indicates  which  high-level  ICC  Mission  Function  (the  highest  level  requirement) 
pertains  to  each  issue. 

Ultimately,  the  connection  of  analysis  results  to  ICC  requirements  will  help  the  ICC 
establish  a  starting  point  for  their  assessment.  DEP  has  agreed  to  deliver  an 
assessment  of  ICC  requirements  based  on  the  table  above. 
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RecommendationsJI  of  2) 

1  The  ICC  and  DEP  need  more  collaboration  to  define  roles  of  stakeholders 
during  the  test  planning  phase 

•  ICC  should  consult  DEP  Functional  Leads  during  Test  Planning  Guide  preparation 

•  DEP  Functional  Leads  should  participate  in  ICC  assessment  brief  or  should 
receive  this  brief  from  prior  events  so  Leads  can  understand  utility  of  test  results 

2  All  stakeholders  should  agree  on  definitions  of  key  terms  to  promote  a 
unified/consistent  understanding  of  testing  terminology 

•  Test  Case  Test  Objective,  ICC  Test  Requirement,  etc. 

3.  TP&E  should  work  with  DM&A  and  ICC  to  develop  the  test  planning  guide 

•  Develop  the  test  objectives  to  explicitly  address  the  ICC  requirements 

•  The  order  of  the  test  case  execution  should  be  driven  by  priorities  based  on  ICC 
requirement  trace  and  the  test  configuration  changes 

4  Network  architecture  diagrams  should  be  updated  and  distributed  after  the 
execution  of  an  event  to  reflect  actual  testing  conditions 

5  All  participants  should  adhere  to  planned  milestones  for  each  event 

CNA 


After  participating  in  several  process  improvement  events,  discussing  results  with 
ICC  and  DEP  stakeholders,  and  completing  the  BUR,  we  have  compiled  a  list  of 
recommendations  for  the  ICC  and  DEP  to  help  incorporate  the  above  results. 

(1)  The  ICC  and  DEP  need  more  collaboration  to  define  roles  of  stakeholders  during 
the  formal  test  planning  phase.  The  ICC  must  consult  with  DEP  Functional  Leads 
while  preparing  the  formal  Test  Planning  Guide  to  determine  the  availability  and 
feasibility  of  testing  and  analysis  needs  for  a  given  event.  To  that  end,  the  DEP 
Functional  Leads  should  participate  in  the  final  ICC  assessment  brief  to  the  fleet 
after  each  interoperability  assessment  event  (or  at  minimum  receive  a  copy  of  this 
brief)  so  they  can  better  plan  for  upcoming  events  based  on  the  current  needs  of  the 
ICC.  In  this  capacity,  DEP  will  be  able  to  provide  better  support  to  the  ICC. 

(2)  The  ICC  and  DEP  functional  leads  must  come  to  a  consensus  and  finalize  the 
definition  of  key  terms  (i.e.,  Test  Case,  Test  Requirement,  Functional  Requirement, 
etc.)  to  promote  a  unified,  consistent  understanding  of  key  interoperability 
assessment  testing  terminology.  At  present,  based  on  conversations  with  personnel 
in  both  ICC  and  DEP,  we  believe  there  may  be  subtle  inconsistencies  and 
differences  in  interpretations  of  these  terms  that  create  confusion  between  the 
organizations.  While  this  may  appear  to  be  a  matter  of  semantics,  in  fact,  it  was  one 
of  the  major  hurdles  we  faced  when  conducting  the  BUR. 
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(continued) 


(3)  TP&E,  DM&A,  and  ICC  must  work  together  to  develop  the  formal  test  planning 
guide.  Currently,  some  of  the  test  objective  statements  in  the  informal  ICC  test 
planning  guide,  such  as  “compare  SIAP  results  of  TSG  to  RSG”,  are  vague  and  do 
not  clearly  connect  with  ICC  requirements.  Therefore,  these  test  objectives  need  to 
be  written  in  a  way  to  indicate  their  connections  to  the  ICC  requirements.  Once  the 
motivation  and  intention  of  test  objectives  are  clear,  DEP  and  ICC  can  establish 
testing  and  analysis  priorities. 

In  addition,  the  order  of  the  test  case  execution  should  be  driven  by  two  factors:  the 
priorities  based  on  the  ICC  requirement  trace  and  the  minimization  of  test 
configuration  changes. 

(4)  If  modifications  to  planned  testing  conditions  are  made  during  test  execution, 
TP&E  must  update  the  network  architecture  diagrams  and  configuration  tables  after 
event  execution. 

(5)  All  stakeholders  must  adhere  to  planned  milestones  for  each  testing  event. 
Collaboration  has  been  designed  into  the  Future  State,  and  it  is  important  for 
personnel  to  participate  at  all  points  where  they  have  a  role  or  responsibility. 
Because  of  the  early  collaboration  among  the  stakeholders  in  the  RIE  Future  State, 
a  majority  of  test  planning  labor  can  be  accomplished  during  the  initial  ICC  planning 
phase  that  may  alleviate  time  constraints  prior  to  test  execution.  Additional  evolution 
of  the  ICC-DEP  assessment  testing  process  may  need  to  occur  after  the  Future 
State  process  has  been  executed,  but  the  success  of  the  RIE  events  and  BUR  will 
rest  on  personnel  adapting  to  the  new  timelines. 
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Recommendations  (2  of  2) 

6.  Using  the  requirements  trace,  the  ICC  should  review  test  objectives  to 
determine  if  they  are  adequately  addressing  the  test  requirements 

•  Test  Objectives  may  be  condensed  based  on  the  requirements  they 
address 

•  Over-representation/under-representation  of  requirements 

7  The  One-Scenario-Fits-AII  approach  is  not  adequate  to  address  all 
test  cases/test  objectives 

•  ICC  and  DEP  functional  leads  should  consider  specialized  scenario 
development 

•  Inclusion  of  the  DEP  stakeholders  and  the  proposed  17-week  ICC  test 
planning  timeframe  support  the  development  of  an  event-driven  Scenario 
Working  Group 

8.  The  Operational  vs.  Engineering  nature  of  DEP  testing  should  be 
defined 

•  ICC  needs  to  highlight  which  test  cases  are  driven  by  fleet  operations  or 
by  engineering  needs  (e  g  SIAP  1  &  2) 

CNA 


(6)  Using  the  requirements  trace,  the  ICC  must  review  test  objectives  on  an  event 
by  event  basis.  The  requirements  trace  matrix  highlighted  that  the  efficiency  of 
testing  may  not  be  maximized.  There  is  overrepresentation  of  some  requirements 
and  underrepresentation  of  others.  Several  test  objectives  map  directly  to  the  same 
set  of  requirements,  and  it  may  be  possible  for  the  ICC  to  consolidate  these.  Other 
test  objectives  can  be  reconsidered  because  they  only  indirectly  address  ICC 
requirements. 

(7)  When  comparing  the  ICC  requirements  to  both  the  current  and  proposed 
analysis  methodology,  we  determined  that  the  One-Scenario-Fits-AII  approach  is 
not  adequate  to  address  all  test  objectives.  The  ICC  and  DEP  Functional  Leads 
should  consider  more  specialized  scenario  development,  perhaps  per  Test  Case  if 
necessary.  The  proposed  17-week  ICC  test  planning  period  and  early  inclusion  of 
the  DEP  stakeholders  in  planning  support  the  development  of  an  event-driven 
Scenario  Working  Group  that  reviews  planned  test  cases  and  the  required  scenario. 
Alternately,  adding  a  few  specialized  tracks  to  the  current  Air  Defense  Exercise 
(ADEX)  scenario  to  suit  specific  test  cases  could  increase  the  analytical  utility  of  this 
commonly  used  scenario. 

(8)  The  ICC  needs  to  highlight  which  test  cases  are  operationally-  or  engineering- 
driven.  For  example,  we  believe  the  majority  of  the  test  cases  are  intended  to 
replicate  operational  situations  faced  by  the  Fleet.  However,  the  ICC  often  requests 
test  cases  that  are  meant  to  root  cause  specific  issues  between 
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(continued) 

combat  systems  that  have  been  observed  over  multiple  interoperability  assessment 
test  events.  These  test  cases  are  more  engineering  in  nature  and  are  not 
necessarily  tied  to  specific  Fleet  Platform  Certification  Decisions  (PCD).  ICC  Test 
Management  and  ICC  Assessment  must  communicate  the  engineering  and/or 
operational  nature  during  the  creation  of  the  formal  test  planning  guide. 
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Way  forward 

•  DEP  and  ICC  should  conduct  an  implementation  working  group 
based  on  the  outcome  of  the  BUR 

Many  solutions  to  process  issues  have  been  proposed  but  specific 
implementation  and  organizational  changes  have  not  been  completed 

•  Select  a  specific  test  event  in  FY09  to  implement  process 
improvements 

CG  Mod  10  DEV  (3rd  event)  is  first  opportunity  due  to  planning  timeline 


CNA 


This  annotated  brief  fulfills  the  requirements  of  the  BUR  as  outlined  in  the  FY08 
Strategic  Plan  and  serves  as  the  delivered  report.  We  believe  the  ICC  and  DEP 
stakeholders  can  easily  use  the  products  contained  within  this  report  and 
incorporate  them  into  a  collaborative  and  formalized  Test  Planning  Document.  The 
ICC  has  accepted  both  the  proposed  requirements  trace  and  the  roles  and 
responsibilities.  DEP  functional  leads  also  contributed  comments  and  have  agreed 
to  the  process. 

Based  on  the  outcome  of  the  BUR,  the  DEP  and  ICC  should  conduct  an 
implementation  working  group.  Many  solutions  to  process  issues  have  been 
proposed,  but  specific  implementation  and  organizational  changes  have  not  been 
completed.  Without  a  final  face-to-face  working  group  to  decide  how  to  best 
implement  all  of  the  process  changes  developed  during  the  past  year,  it  is  highly 
probable  that  the  organizational  problems  that  necessitated  the  BUR  will  persist. 
After  the  implementation  working  group,  the  ICC  and  DEP  program  management 
should  target  a  test  event  to  begin  implementing  the  process  changes  discussed  in 
the  BUR.  Based  on  the  new  extended  planning  timeline,  we  have  already  passed 
the  time  when  planning  would  need  to  begin  for  the  first  two  proposed  test  events  in 
FY09.  Therefore,  we  recommend  that  the  third  event  in  FY09  (the  CG  Mod  10  DEV 
test)  will  be  the  first  opportunity  to  fully  implement  the  BUR. 

After  the  completion  of  several  interoperability  assessment  tests,  DEP  and  ICC 
should  revisit  the  process  changes  introduced  by  the  BUR  to  determine  the 
effectiveness  of  the  proposals.  For  example,  the  full  17-week  ICC  planning  period 
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(continued) 

may  not  be  necessary  for  every  event  in  a  calendar  year.  Process  improvement  and 
development  should  be  an  ongoing,  continuous  effort. 
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Appendix  A 


•  Roles  and  Responsibilities 
ICC  Planning  Period 
ICC  Primary  Stakeholder 


CNA 


This  appendix  contains  the  recommended  roles  and  responsibilities  for  the  ICC 
planning  portion  of  the  future  process  state.  Each  step  contains  a  list  of 
stakeholders  who  are  responsible  for  completing  an  action.  One  of  those 
stakeholders  is  designated  as  accountable  for  making  sure  the  action  is  complete. 
We  also  suggest  personnel  who  should  be  consulted.  This  list  contains  stakeholders 
who  will  be  affected  by  the  outcome  of  the  step. 

Both  this  appendix  and  Appendix  B  are  adaptations  of  the  RACI  charts  that  are 
developed  in  Lean  Six  Sigma  process  improvement  events.  RACI  stands  for 
Responsible,  Accountable,  Consulted,  and  Informed.  Typically,  participants  should 
complete  these  charts  for  a  newly  designed  process  before  concluding  the  event. 
However,  time  did  not  allow  for  the  completion  of  these  charts  at  the  ICC-DM&A 
Rapid  Improvement  Event.  Therefore,  we  suggest  the  roles  and  responsibilities 
here. 
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Define  Test  Requirement  (ICC) 


Pull  Functional 
Requirements 

Action  Required  By 

Consultation  with 

ICC  Test  Management* 

ICC  Requirements 

DEP  Program  Management 

I   

Identify  Test 
Objectives 

Action  required  by 

Consultation  with 

ICC  Test  Management* 

ICC  Requirements 

ICC  Assessment 

DEP  Functional  Leads 

1 

Pull  Test 
Conditions  and 
Configurations 

Action  required  by 

Consultation  with 

ICC  Requirements 

ICC  Test  Management* 

ICC  Assessment 

DEP  Functional  Leads 

■  

Define 

Stakeholders  for 
Test  Planning 
Guide 

Action  required  by 

Consultation  with 

ICC  Test  Management 

DEP  Program  Management* 

DEP  Functional  Leads 

ICC  Requirements 

ICC  Assessment 
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(*)  Indicates  party  held  accountable  for  action 


Define  Test  Requirement  (ICC) 
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ICC  Test  Management* 
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Systems  in  the  DEP 
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ICC  Test  Management* 

ICC  Assessment 

SSA/SME 

All  DEP  Functional  Leads 

in 

1 

Develop  List  of  Core 
DEP  Test  Objectives 
+  Configuration 
Requirements 

Action  required  by 

Consultation  with 

ICC  Test  Management* 

ICC  Requirements 

ICC  Assessment 

All  DEP  Functional  Leads 

ICC  Safety 

E&O  Lead 
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(*)  Indicates  party  held  accountable  for  action 


Milestone 

Progress 

> 

♦ 

00 

— ►  5 

</) 

w 

j 

w 

w 

V 

w 

Y 

? 

w 

f 
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Test  Planning 


CNA 


Develop  Test 
Cases  for 
Functional  Reqs 

Action  required  by 

Consultation  with 

ICC  Test  Management* 

ICC  Assessment 

ICC  Requirements 

All  DEP  Functional  Leads 

■ 

Develop  Test 
Configuration 

Action  required  by 

Consultation  with 

ICC  Test  Management* 

TP&E  Lead 

ICC  Assessment 

DM&A  Lead 

E&O  Lead 

1 

Develop  Setup 
Requirements 

Action  required  by 

Consultation  with 

ICC  Test  Management* 

TP&E  Lead 

E&O  Lead 

ICC  Assessment 

DM&A  Lead 

1   - 

Develop 

Assessment 

Methodology 

Action  required  by 

Consultation  with 

ICC  Assessment* 

DM&A  Deputy 

ICC  Test  Management 

ICC  Requirements 

(*)  Indicates  party  held  accountable  for  action 


W 

w 

w 

w 


Test  Planning 


Distribute  Guide  to 
ICC  Players 

Action  required  by 

Consultation  with 

ICC  Test  Management* 

ICC  Stakeholders 

1 

Develop  Test 
Document 

Action  required  by 

Consultation  with 

ICC  Test  Management* 

ICC  Assessment 

ICC  Requirements 

DEP  Functional  Leads 

1 

Review  ICC 
Distribution  List 

Action  required  by 

Consultation  with 

ICC  Test  Management* 

ICC  Stakeholders 

CNA 


(*)  Indicates  party  held  accountable  for  action 
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Test  Planning 


Test  Planning 


D 


Collaborate  with 
DM&A  to  Develop 
Analysis 
Methodology 

Action  required  by 

Consultation  with 

ICC  Assessment* 

ICC  Test  Management 

DM&A  ACB  Lead 

TP&E  Lead 

E&O  Lead 

, - w 

Incorporate  DM&A 

Comments 

Coordinate  with 
SMEs  to  Support 
TO  Assessment 

Action  required  by 

Consultation  with 

DM&A  Lead* 

ACB  Lead 

DEP  Program  Manager 

DEP  Functional  Leads 

ICC  Assessment 

_ 7?  ICC/DEP 

Distribute  to  DEP  j  Functional  Lead 

TP&E  for  TPWG  |  Primary  Role 

-  Handover 


CNA 


(*)  Indicates  party  held  accountable  for  action 
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Appendix  B 

•  Roles  and  Responsibilities 

DEP  Planning/Execution/Analysis  Period 

DEP  Primary  Stakeholder 

CNA 

This  appendix  focuses  on  roles  and  responsibilities  during  the  portion  of  the  Future 
State  when  DEP  is  a  primary  stakeholder. 
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DEP  Planning 


Review  Test 
Planning  Guide 

Action  Required  By 

Consultation  with 

All  DEP  Functional  Leads  (*) 

ICC  Test  Management 

ICC  Assessment 

■ 

Develop  Schedule 
&  Resources 

Action  required  by 

Consultation  with 

All  DEP  Functional  Leads  (*) 

DEP  PM 

■ 

Develop  POA&M 

Action  required  by 

Consultation  with 

All  DEP  Functional  Leads  (*) 

DEP  PM 

ICC  Assessment 

■ 

Assign  TPWG 
Dates 

Action  required  by 

Consultation  with 

TP&E  Lead  (*) 

DM&A  Lead 

E&O  Lead 

1 

Test  Data  (RMT  & 
Execution) 

Action  required  by 

Consultation  with 

TP&E  Lead  (*) 

E&O  Lead  (*) 

DEP  PM 

DM&A  Lead 

ICC  Assessment 

CNA 


(*)  Indicates  party  held  accountable  for  action 


Milestone 

Progress 

> 

ro 

5 

X 

V) 

W 

W 

\  ; 

y 

T 

w 

f 

Analysis  Planning  (DM&A) 


Hold  TPWG  #1 

Action  required  by 

Consultation  with 

TP&E  Lead  (*) 

All  DEP  Functional  Leads 

ICC  Test  Management 

ICC  Assessment 

1 

Discuss  Analysis 
Methodologies  & 
Data 

Action  required  by 

Consultation  with 

DM&A  Deputy  (*) 

ACB  Lead 

ICC  Assessment 

DM&A  Data  Manager 

8 

Milestone 

Progress 


t 


5 

x 

to 


W 

y 

W 

y 


▼ 


CNA 


(*)  Indicates  party  held  accountable  for  action 


V 


>r 
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Analysis  Planning  (DM&A) 


Milestone 

Progress 

Action  required  by 

Consultation  with 

♦ 

uonaDoraie  Detween 

ICC/DEP  on  TOn"Cs 

ACB  Lead  (*) 

ICC  Assessment 

DM&A  Lead 

NJ 

1 

r 

5 

Action  required  by 

Consultation  with 

<s> 

Develop  TO 
Assessment  Matnx 

ACB  Lead  (*) 

ICC  Assessment 

DM&A  Deputy/Lead 

W 

t 

Identify  Data  to 
Support  TOs  & 
TCs 

Action  required  by 

Consultation  with 

w 

ACB  Lead  (*) 

ICC  Assessment 

DM&A  Deputy/Lead 

Develop 

i  TO/TC 

Action  required  by 

Consultation  with 

w 

Assessment 

Methodology 

ACB  Lead 

ICC  Assessment  (*) 

DM&A  Deputy/Lead 

NA 

w 

(*)  Indicates  party  held  accountable  for  action 

> 

f 

Test  Planning  (TP&E) 


Hold  TPWG  #1 

Action  required  by 

Consultation  with 

TP&E  Lead  (*) 

All  DEP  Functional  Leads 

ICC  Test  Management 

1 

Discuss  TCs 

Action  required  by 

Consultation  with 

ICC  Test  Management 

All  DEP  Functional  Leads 

1  

Review  Sites  and 
Baselines 

Action  required  by 

Consultation  with 

TP&E  Lead  (*) 

E&O  Lead  (*) 

ICC  Test  Management 

DM&A  Deputy/Lead 

i 

Review 

Architecture,  Links. 
Filters,  and  IDs 

Action  required  by 

Consultation  with 

All  DEP  Functional  Leads(*) 

ICC  Assessment 

ICC  Test  Management 

CNA 


{*)  Indicates  party  held  accountable  for  action 


Milestone 

Progress 


ho 


w 

w 

W 

w 


T 
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Test  Planning  (TP&E) 


“Confirm" 
Identify  TCs 


CNA 


(*)  Indicates  party  held  accountable  for  action 


Action  required  by 

Consultation  with 

TP&E  Lead  (*) 

Other  DEP  Functional  Areas 

“Confirm" 

Identify  Platforms 

Action  required  by 

Consultation  with 

TP&E  Lead  (*) 

E&O  Lead 

ICC  Test  Management 

1 

“Confirm" 

Identify  Data 
Requirements 

Action  required  by 

Consultation  with 

DM&A  Lead  (*) 

E&O  Lead  (*) 

i 

“Confirm" 

Identify 

Architecture.  Links. 
Filters,  and  IDs 

Action  required  by 

Consultation  with 

All  DEP  Functional  Leads 

ICC  Test  Management 

ICC  Assessment 

Test  Planning  (TP&E) 
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Test  Planning  (TP&E) 


Develop  2'*  Draft 
of  Test  Procedures 

Action  required  by 

Consultation  with 

Test  Director  (*) 

ICC  Test  Management 

TP&E  Lead 

i 

Conduct  Peer 
Review 

Action  required  by 

Consultation  with 

Test  Director  (*) 

All  DEP  Function  Leads 

ICC  Assessment 

ICC  Test  Management 

1 

Hold  TPWG  #3 

Action  required  by 

Consultation  with 

Test  Director  (*) 

All  DEP  Functional  Leads 

ICC  Assessment 

ICC  Test  Management 

Adjudicate 

Comments 

Action  required  by 

Consultation  with 

Test  Director  (*) 

ICC  Test  Management 

ICC  Assessment 

All  DEP  Functional  Leads 

CNA 


(*)  Indicates  party  held  accountable  for  action 


Test  Planning  (TP&E) 


CNA 


Action  required  by 

Consultation  with 

Conduct  RMT 

Test  Director  (*) 

E&O  DEP  Operations  Leadf) 

ICC  Test  Management 

ACB  Lead 

1 

Action  required  by 

Consultation  with 

Validate  Data 

ACB  Lead  (*) 

ICC  Test  Management 

ICC  Assessment 

E&O  DEP  Operations  Lead 

1 

Action  required  by 

Consultation  with 

Validate  SIM/STIM 

E&O  DEP  Operations  Leadf) 

ACB  Lead 

Test  Director 

■ 

Action  required  by 

Consultation  with 

Run  Part  of  TCs 

Test  Director  (') 

E&O  DEP  Operations  Lead 

ICC  Test  Management 

ICC  Assessment 

ACB  Lead 

Milestone 

Progress 


(*)  Indicates  party  held  accountable  for  action 


£ 


Fulfills 

Milestone 


w 
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Test  Planning  (TP&E) 
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Appendix  C 


•  Sample  requirements  trace  for  all  test  cases  in  SSDS  10  DEV  event 


CNA 


Appendix  C  contains  examples  of  how  to  demonstrate  traceability  of  ICC 
requirements  through  the  Test  Objectives.  The  tools  we  used  are:  the  TADIL-CEC 
architecture  map,  the  test  configuration  matrix,  and  the  ICC  requirements  trace.  We 
split  the  requirements  trace  table  over  two  slides  for  easier  viewing.  We  have  used 
the  SSDS  10  DEV  08  event  as  the  basis  for  these  examples  and  provide  here  all 
nine  Test  Cases.  We  did  not  attempt  to  trace  ICC  requirements  to  Test  Objectives 
that  refer  to  GCCS-M  because  currently  there  are  no  ICC  requirements  for  GCCS- 
M. 
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Test  Case  1 .  Test  Strike  Group  1 


TAD1L-CEC  architecture 


CNA 
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Test  Case  1 .  Test  Strike  Group  1 

Configuration  Matrix 

—  • 

T*ml  CM*  1 

Link  Statu* 

Correlation 

Platform 

DTQ 

L-11 

L-16 

STJ 

CECDDS 

Of 

L-11  TN  ASSIGN 

TPF 

GRU 

BFCC 

ssos 

fi 

0 

A 

0 

E 

D 

D 

E 

D 

CVO 

AWS  6  '  7 

5 

0 

A 

E 

D 

E 

D 

E 

AWS  5  3  9 

11 

A 

A 

N/A 

E 

E 

0 

E 

E 

ACOS 

5 

A 

D 

D 

Hi  A 

D 

0 

E 

0 

E 

E2C 

5 

0 

A 

0 

E 

D 

N/A 

E 

N/A 

E 

|  A  -  Active  D  =  Disabled  D/O  =  Default  On  E  =  Enabled  N  A  =  Not  Applicable  | 

CNA 

Test  case  1.  Test  Strike  Group  (1  of  2) 

ICC  Requirements  Trace 


Tati  Cm* 

 ipafAt  ICC  Maatar  U.«  Aamaraawnn 

■1  tr. 

n«a  Track 

•mam 

| 

l 

l 

1 

f 

it 

l! 

i 

J 

i 

J 

li 

Q 

COO 

i 

\ 

t 

2 

t 

2 

t 

2 

5 

> 

5 

j 

i 

e 

s 

t 

£ 

? 

0 

3 

3 

5 

8 

s 

S 

Taat  Cm—  1 

7mm*  *r*a  Or  ai# 

i  Com^va  S4AP  fMirta  »  *S6 

0 

6 

0 

1 

* 

• 

D 

D 

1  Va-*»  tna  SSOS  *  aM  to  taw  am  •mr  tn  k  anaa 

and  qW  >ntaropa>aMrty  In  batad  an  SSOS  and 
rXMS  VDOi  v  Xa» 

D 

0 

D 

D 

D 

D 

D 

D 

D 

D 

D 

D 

D 

D 

D 

D 

D 

D 

D 

AAW  Kt/Aomi  an  |  C  T  P  tor  -/m  SC  .  AOR 

to  ma  COP  via  OCCS^ICST  LINK 

h  Vanfy  that  tha  platform  unda<  tatf  arimfl  am  no6m 

tm—<l  qn  mi^ad  -aporlirtg  tartar*  and  »-♦* 

•  Va^y  trial  tria  platform  undar  aat  acting  a*  Pa*a<*  noda 
cao  ayppexi  na  CTP  Marmar  a  a*»*y  to  proparty  tota  a* 

•nan  on*  on.  urd  ,  at  L«»  «n*>o<  »«* 

?  Va*#y  tna  GCCS-M  a*  tna  5*>S  platform  undar  ia*i  proparty 
-acamaaand  pnxataai  r*-i|  data  toyn  ADS  *ai  t*a  MTC  '^aoe 

CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  1 .  Test  Strike  Group  (2  of  2) 


jCC  Requirements  Trace 


TmI  Chi 

ObJscttvM 

DEP 

Specific 

Requtremtnts 

HON -DEP 
SpacHlc 
Raqutremant* 

FfttrKtty  DaconfWcffOrt 

Forca  Coordtaefon 

12/15/11  Bit  HJ  Hgmt 

o 

a 

Engagement 

Coordination 

Engagement  Status 

Q 

u. 

2 

8 

3 

3 

U 

u 

Ik 

TmI  Cih  1 

Te*t  Sinks  Grout 

1  Compart  SIAP  result  a  to  RSG 

*2  Verify  the  ability  of  SSOS  lo  operate  within  lh«  representativtf  SG 

3  Vanty  Iht  SSOS  ta  a t»r  lo  operate  with  High  TN  IU  unct- 
aod  other  nltroparabtkty  hxs*  based  on  SSOS  and 

CDiVS  VDO»  or  fix  W»l» 

D 

0 

D 

*  Examnt  Iht  ability  of  tot  SG  to  track  and  report  Iht 

AAW  situational  awareness  and  CTP  lor  (he  SG's  AOR 
to  too  COP  via  GCCS-M/CST  and  LINK  networks 

D 

5  Vaitfy  that  toe  platform  under  let!  acting  aa Parent  nod, 
l*  able  to  disseminate  and  updala  at  track*  to  to#  TOP  COP 
based  on  assigned  rsportmg  sector*  and  mission* 

D 

6  Vanty  that  tot  platform  under  Loti  acting  aa  Parant  nod# 
can  support  tot  CTP  Mantgtr's  ab*ty  lo  property  fu»e  air 
track  data  horn  chad  nodes  and  upon  jpdalt  tracks  to  toe 

TOP  COP  ba*ad  on  sptcifttd  mtmmion  requirements 
when  onty  ont  unit  t*  assigned  as  Link  tnpu;  Ship 

D 

7  Verity  toe  GCCS-W  at  toe  SSOS  platform  under  lest  properly  receive* 
and  practise*  Lmklfi  data  from  A  OS  1  via  the  MTC  interface 

D 

*  OvtrtM  DEP  Isit  otofscttvs 

CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  2.  Test  Strike  Group  with  SGS  6.0. 7. 5 


Test  case  2.  Test  Strike  Group  with  SGS  6.0.7.5 

Configuration  Matrix 


A  =  Active  D  =  Disabled  D/O  =  Default  On  E  =  Enabled  N/A  =  Not  Applicable 


CNA 
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Test  case  2.  Test  Strike  Group  with  SGS  6.0. 7.5  M  of  2^ 

jCCLRequirements  Trace 


Tut  Cm 

Mali 

at  Tr 

Krtohfl 

*urvtol 

Id# 

nOfkalkm 

1 

! 

4 

* 

f 

l 

3 

]f 

] 

f 

| 

i 

ii 

e 
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5 

5 

£ 

* 

5 

p 

i 
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£ 

P 

99 

P 

5 

S 

3 

3 

fi 

3 

c 

fi 

Tm!  Ciu  2 

T##t  Sink#  Group 
with  SGS 
8075 

1  Gomp#f»  SLAP  rotuKs  from  lh#  RSG  #nd  TSG  1  wSh  TSG  2 

at  SOS' AC  6  0  7  5  •*  SSOS  AWS  5  3  #  #nd  AWS  6  1  7 

Dfl#r»nc#«  IVw  lh#  group#  #nd  #ny  »nofn#k##  frwn  ob##rv#lior 
or  dat#  will  0#  u *#d  to  qo#u#  d#l#ii#d  #o#iy*<« 

0 

D 

0 

1 

■ 

■ 

D 

D 

2  V#fTfy  c»fT#4#lionAl#00'T#L«|(on  proc##mtng  and  oth«f 
improvtroanu  t»e*#d  or  th#  SGS  VDC  Of  icpicaM#  lx  H 

b 

c 

D 

1 

' 

1 

' 

’ 

3  V#fity  thal  lh#  plaMpon  und#f  t##l  acliog  a#  Parant  nod# 
can  tuppofl  lh#  CTP  Managaf’x  ability  o  prpp#rty  fua#  wr 
track  data  from  chHd  noda#  and  ••port  jpdat#  track*  to  lh# 

TOP  COP  baaad  on  »paeAad  wnwr  raqc  raroanti  wn#n 
mpr#  than  pna  un*  #  a»^nad  a#  l»nk  inpul  Sf*p 

CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/ objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 


Test  case  2.  Test  Strike  Group  with  SGS  6.0.7.5  (2  of  2) _ . 


ICC  Requirements  Trace 


TMtC«M 

Objective# 

DEP 

Specific 

Requirements 

NON-OEP 

Specific 

Requirements 

Friendly  Deconflictlon 

Force  Coordinstlon 

12/15/19  Bit  lUMgmt 

GCCS 

Engagement 

Coordinstion 

Engagement 

Status 

O 

FD4 

2 

GCCS 

U 

O 

Test  Case  2. 
Tesl  Strike  Group 
with  SGS 

6.0. 7. 5 

1  Compare  SIAP  results  from  Ihe  RSG  and  TSG  1  with  TSG  2 
assess  any  differences  in  performance  based  on  the  incorporation 
of  SGS/AC  6  0.7.5  at  SSOS.  AWS  5  3.9  and  AWS  6. 1.7 

Differences  b'w  the  groups  and  any  anomalies  from  observation 
or  data  will  be  used  to  Queue  delated  analysis 

1 

2  Verify  correiabon/decorreiation  processing  and  other 
improvements  ba^ed  on  the  SGS  VDD  or  applicable  fix  list. 

3  Verify  that  the  platform  under  test  acting  as  Parent  node 

Can  support  the  CTP  Manager's  ability  to  properly  fuse  eir 
track  data  from  child  nodes  snd  report/update  tracks  lo  the 

TOP  COP  based  on  specified  mission  requirements  when 
more  then  one  unit  is  assigned  as  Link  ln£utSbjp^ 

D 

CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  3.  Reference  Strike  Group 
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Test  case  3.  Reference  Strike  Group 

Configuration  Matrix 

1 

Tast  mm  3 

Link  Statu* 

Correlation 

Platform 

DTD 

L-11 

L-11 

STJ 

CEC  DOS 

DF 

L-11  TN  ASSIGN 

TPF 

GRU 

BFCC 

SSDS 

8 

D 

A 

D 

6 

D 

D 

e 

D 
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5 

D 

* 

D 

E 

D 

D 

E 

D 

E 

AWS530 

11 

A 

* 

D 

N/A 

E 

E 

D 

E 

E 

ACDS 

5 

A 

D 

D 

N/A 

D 

D 

E 

D 

E 

E  2C 

5 

D 

A 

D 

E 

D 

N/A 

E 

N/A 

E 

CN 

A  =  Active  D  =  Disabled  D/0  =  Default  On  E  =  Enabled  N/A  =  Not  Applicable 

IA 

Test  case  3.  Reference  Strike  Group  (1  of  2) 


ICC  Requirements  Trace 
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CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  3.  Reference  Strike  Group  (2j)f  2) 


ICC  Requirements  Trace 


DEP 

Specific 

Requirements 

NON-DEP 

Specific 

Requirements 

Friendly  Oeconfliction 

Force 

Coordination 

Teat  Ca»* 

Objectives 

12/15/19  Bit  IU  Mgmt 
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1  Compare  SIAP  results  to  TSG  1  and  2 

1 

Test  Case  3 

2  Examine  the  eWity  of  AWS  5  3  9  basefcne  v*  CDLMS  3  4  4  2-4 
to  operate  wl  the  representative  Strike  Group  AWS  5.3  9  will  be 
operating  with  a  new  version  of  CDLMS  thet  mcluder  corrections 
to  Ht/Lo  TN  manegement 

D 

• 

RSG 

3  Examine  the  ability  of  the  SG  to  track  end  report  the 

AAW  situation  el  ewereness  and  CTP  for  the  SG's  AOR 
to  the  COP  via  GCCS-M/CST  and  LINK  networks 

D 

4  Verify  that  the  platform  under  lest  ectiog  as  FOTC  can  properly 
fuse  air  track  data  from  participant  nodes  end  report/update  tracks 
to  the  TOP  COP  based  on  specified  mission  requirements  when 
onty  one  unit  is  assigned  es  Link  input  Ship. 

D 

5  Determine  eny  differences  between  GCCS-M  COPr  when 
operating  m  CTP  Mensger  and  FOTC  modes  by  comparing 

TSG  1  end  RSG 

D 

CNA 


Direct  (D)  -  data  collected  to  sup{X>rt  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  4.  Maritime  Reporting 


CNA 


Test  case  4.  Maritime  Reporting 

Configuration  Matrix 


I  A  =  Active  D  =  Disabled 


D/O  =  Default  On 


E  -  Enabled 


N/A  =  Not  Applicable 


CNA 


70 


Test  case  4.  Maritime  Reporting  (1  of  2) 


ICC  Requirements  Trace 
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Direct  <D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  4.  Maritime  Reporting  (2  of  2) 


I C C  Requiremen ts  Tra c e 


DCP 
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Requirements 
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Specific 
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FC3 

1  Examine  the  ebifrty  ol  the  SG  to  track  and  report  the  maritime 
situational  awareness  and  CTP  lor  Ihe  SG's  AOR  to  the  COP 
via  GCCS-M/CST  end  LINK  networks 

D 

2  Verify  the  ability  of  the  SG  to  property  filter  er  and  surface  tracks 
on  GCCS-M  network 

D 

Test  Case  4 
Mantime  Reporting 

3.  Verify  that  the  platform  under  test  acting  as  Parent  node 
is  eble  to  accept  disseminata  and  update  surface  tracks  to  the 

TOP  COP  based  on  os  started  reportina  sectors  and  missions 

D 

4  Venfy  thal  Ihe  platform  under  test  e cling  as  parent  node  is  eble 
can  support  the  CTP  manager's  eMity  to  properly  fuse  surface 
track  data  from  child  nodes  end  report/update  tracks  to  the 

TOP  COP  based  on  specified  mission  requrements  when  more  then 
one  unit  is  assigned  as  link  Inool  Ship 

0 

5.  Venfy  SUW  Force  Order  engagement  and  weapons  status 
indications  across  TADIl  networks 

O’- 

D" 

■’  NON-DEP  ICC  Master  list  requirements 

CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  5.  SIAP  Assessment  1 


TADIL-CEC  architecture 


CNA 
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— 
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Data  Silent 
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Data  Forwarder 

■  ^ACDS^ 


72 


Test  case  5.  SIAP  Assessment  1 

Configuration  Matrix 
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|  A  =  Active  D  =  Disabled  D/O  =  Default  On  E  =  Enabled  N  A  =  Not  Applicable  DS  =  Data  Silent 

CNA 

Test  case  5.  SIAP  Assessment  1  (1  of  2) 


ICC  Requirements  Trace 


CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case /objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  5.  SIAP  Assessment  1  (2  of  2) 


ICC  Requirements  Trace 
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Test  Case  5 

1 .  Calculete  SIAP  metncs  to  use  as  a  reference  to  compare  results 

SIAP 

b/w  this  test  case  end  TSG  1 . 

D 

D 

D 

Assessment  1 

Assist  in  root-ca  using  E2C  (Data  Silent)  track  reporting  issues 

observed  during  previous  interoperability  certification  test  events 

Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)-  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 

CNA 
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Test  case  6.  SIAP  Assessment  2 

-  • 

TADIL-CEC  architecture 
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Data  Forwarder 

Test  case  6.  SIAP  Assessment  2 

Configuration  Matrix 
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|  A  =  Active  D  =  Disabled  D/O  =  Default  On  E  =  Enabled  N/A  =  Not  Applicable  DS  =  Data  Silent  | 

CNA 
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Test  case  6.  SIAP  Assessment  2  (1  of  2) 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 


CNA 


Test  case  6. SIAP  Assessment  2  (2  of  2) 

ICC  Requirements  Trace 
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D 
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and  Block  0  (Data  Siienl)  track  reporting  issues 

observed  during  previous  inleroperabilily  certificalion  lesl  events 

Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)-  data  collected  to  support  a  specific  test 
case/ objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 


CNA 
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Test  case  7.  Concurrent  OP  1  (Link) 


TADIL-CEC  architecture 


CNA 
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Test  case  7.  Concurrent  OP  1  (Link) 

Configuration  Matrix 
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Test  case  7.  Concurrent  OP  1  (Link)  (1  of  2) 
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Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 


CNA 
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Test  case  7.  Concurrent  OP  1  (Link)  (2  of  2) 


ICC  Requirements  Trace 
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Test  Case  7 
Concurrent  Ops  1 
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///////////A 

6  Examine  the  ability  ot  the  SG  to  track  and  report  the 

AAW  situational  awareness  and  CTP  for  the  SG's  AOR 
to  the  COP  via  GCCS-M/CST  when  Parent  and  Child  nodes  are 
operating  on  the  two  disparate  data  link  networks  within  the  same 
area  of  operation 

D 

7  Verify  that  the  platform  under  test  ecting  as  Child  node  is  eble  to 
disseminate  and  update  tracks  to  the  Parent  node  based  on  assigned 
raporting  sectors  end  missions 

"iT 

~ 

8  Examine  the  ability  of  the  piatfonm  under  test  Child  node  to  support 
the  CTP  Manager1-  ability  to  properly  fuse  air  track  data  when  Parent 
and  Chid  nodes  are  operating  on  two  disparate  data  link  networks 
within  the  same  area  of  operation 

1 

y////////////z 

Cen  not  be  tested  in  final  test  procedures 

1 

CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)-  data  collected  to  support  a  specific  test 
case/ objective  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  8.  Concurrent  OP  2 


TADIL-CEC  architecture 


CNA 


^ACDS^ 
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CEC 

—  •  — 

Link-11 
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Link-11  (2) 

Data  Forwarder 

Test  case  8.  Concurrent  OP  2 


A  =  Active  D  =  Disabled  D/O  =  Default  On  E  =  Enabled  N/A  =  Not  Applicable  DS  =  Data  Silent 


CNA 
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Test  case  8.  Concurrent  OP  2  (1  of  2) 


ICC  Requirements  Trace 
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Direct  (D)  -  data  collected  to  support  a  specific  test 
case/ob|ective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/oh)ective.  when  comhined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 


CNA 


Test  case  8.  Concurrent  OP  2  (2  of  2) 


ICC  Requirements  Trace 
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5  Verify  whether  Model  5  C2P  units  overwrite  high  TN  block  when 
assigned  DF  rate  If  track  block  is  overwritten,  what  is  II  replaced  with? 

CNA 


Direct  (D)  -  data  collected  to  support  a  specific  test 
case /objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requiremenl  has  been  met 
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Test  case  9.  SSDS  Data  Forwarding 


TADIL-CEC  architecture 
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Test  case  9.  SSDS  Data  Forwarding 

Configuration  Matrix 

-  • 

T*St  CM*  1 

Link  Stilus 

Correlation 

PH.*™ 

OTQ 

1-11 

L-16 

STJ 

CEC  DOS 

Df 

L-11  TN  ASSIGN 

TPf 

GFU 

BFCC 

SSOS 

A 

A 

A 

E 

D 

0 

E 

D 

O'O 

AWS  6  I  7 

5 

D 

A 

0 

E 

D 

0 

E 

E 

E 

AWS  S  :t  9 

0 

D 

* 

N'A 

E 

E 

E 

D 

E 

ACDS 

*> 

A 

D 

0 

H< A 

D 

D 

E 

D 

E 

E2C 

5 

0 

A 

0 

D 

D 

N'A 

E 

NIA 

E 

A  Active  D  =  Disabled  D/0  =•  Default  On  E  =  Enabled  N/A  =  Not  Applicable  DS  -  Data  Silent 

CNA 

Test  case  9.  SSDS  Data  Forwarding  (1  of  2) 


ICC  Requirements  Trace 
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Direct  (D)  -  data  collected  to  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DEP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Test  case  9.  SSDS  Data  Forwarding  (2  of  2) 

ICC  Requirements  Trace 


DEP 

Specific 

Requirements 

NON-DEP 

Specific 

Requirement* 

Friandly  Dsconfliction 

Force 

Coordination 

Teat  C*m 

Objective* 

12/1 5/1 9  Bit  IU  Mgmt 

M 

U 

8 

Engagement 

Coordination 

Engagement 

Stetua 

e 

s 

Ik 

s 

c ft 
a 

8 

U 

8 

ik 

1.  Examine  the  CTP  of  e  SG  when  SSDS  is  operating  es  DF  b/w 

Link- 16,  Unk  11,  end  STJ 

D 

2,  Identify  difference*  in  srtuetionel  awareness  b/w  units  operating 
on  separete  networks. 

Test  Cese  9. 

3.  Observe  any  issue*  when  a  SG  operates  with  older  SGS/AC  version* 
end  6.0.7  5  concurrently 

SSDS  Data 
Forwarding 

4.  Verify  SSDS  30  second  lockout  for  CDO  commands  ere  corrected 
et  CDLMS  end  SSDS 

5.  Observe  accurate  AAW  Force  Order  engagement  and  weapons  status 
indications  b/w  TADIL  networks 

D‘" 

D" 

6  Verify  the  GCCS-M  at  the  SSDS  platform  under  test  property  receives 
end  processes  Unk-1 1  deta  from  the  Passive  Link  Tap  (PLT)  inlerfece 

D 

7  Observe  any  difference*  when  GCCS-M  uses  PLT  ve  MTC  interface 
input  at  the  SSDS  platform  by  comparing  with  non-PLT  input. 

D 

*•  NON-DEP  ICC  Master  list  requirements^ 

CNA 


Direct  (D)  -  data  collected  tD  support  a  specific  test 
case/objective  can  be  used  directly  to  address  whether  a 
requirement  has  been  met 

Indirect  (I)  -  data  collected  to  support  a  specific  test 
case/objective,  when  combined  with  other  data  from  a  DFP 
test  or  other  sources  may  be  used  to  address  whether  a 
requirement  has  been  met 
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Appendix  D 

•  ICC  Master  Requirements 

CNA 

This  appendix  contains  the  set  of  requirements  from  the  ICC  Master  Requirements 
list  that  pertain  to  DEP  testing.  We  list  high  level  Functions,  Subfunctions,  and 
specific  Functional  Requirements.  The  following  three  slides  list  the  ICC  master 
requirements  relevant  to  the  DEP  as  determined  by  the  ICC  Requirements  Lead. 
These  interoperability  requirements  are  derived  from  multiple  sources.  At  the  end  of 
the  list,  we  identified  two  additional  ICC  requirements  that  are  not  DEP-specific  but 
can  be  tested  in  the  DEP  environment. 
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DEP  Specif|c  ICC  Master  Requirements 


DEP  SPECIFIC  ICC  Master  Requirments 

Function 

Subfunction 

Functional  Requirement 

Mutual  Tracking 
MT-DEP 

Correlation 

MTS  -  Demonstrate  the  capability  to  perform 
automatic  air  and  surface  correlation 
processing. 

MT6  -  Demonstrate  correct  J 7  2/M 98 
processing  (for  unit  under  test). 

MT7  -  Demonstrate  correct  decorrelation 
processing 

TN  Management 

MT10  -  Verify  each  CEPN  is  associated 
with  one  and  only  one  local  track  (CTSL) 
and  one  end  only  one  Link  track  (LTN)  et  eny 
given  time. 

SLAP 

MT11  *  Evaluate  capability  of  platform  when 
operating  with  the  Strike  Force  to  maintain  e 
single  track  per  object  (SIAP  Clerity). 

MT12  *  Evaluate  capability  of  platform  when 
operating  with  the  Strike  Force  to  maintain  a 
continuous  LTN  and  CEPN  {SIAP  Continuity). 

MT1 3  -  Evaluate  capability  of  pletform  when 
operating  with  the  Strike  Force  to  maintain  a 
common  picture  such  that  the  tracks  held  by 
each  participant  have  the  same  LTN,  LTN/CEPN 
pairing,  ID,  position,  on  the  seme  object 
(SIAP  Commonality). 

CNA 


DEP  Specific  ICC  Master  Requirements 


DEP  SPECtFiC  ICC  Master  Requirments 

Function 

Subfunction 

Functional  Requirement 

Surveillance  Track 
Reporting 
STR-DEP 

Tracking 

ST2  -  Verify  capability  to  trensmit  end  receive 
surveillance  data  on  ell  track  types.  (J  messages) 

Verify  proper  periodicity  of  surveillance  track 
reporting  on  LINK16  and  LINK11.  Verify  entire 
track  block  is  used  before  reuse  of  a  LTN 
a  Verify  that  tracks  received  via  LINK11  orLINK16 
are  displayed  to  the  operator  end  stored  in  the 
combat  system,  b.  Verify  that  tracks  via  LINK1 1/16 
can  be  called  up  {hooked)  by  LTN.  c  Verify  that  all 
local  tracks  eligible  for  link  transmission  ere  being 
transmitted  assuming  TNs  ere  available. 

Translation  and  Data 
Forwerdlng 

ST4  -  Verify  that  all  PPLI  surveillance  track  management 
and  force  status  messages  end  ell  references  to  lUs 
are  properly  translated  end  forwarded  from  LINK  11 
toLINK16,  vice  versa,  and  LINK16to  S-TADILJ  (includes 

12,  15,  and  19  bit  LTN). 

Track  Quality 

ST7  -  Verify  capability  to  determine  horizontal  positional 
accuracy  for  a  circular  area  such  that  there  is  a  95% 
probability  that  the  target  is  within  the  determined  area  at 
the  time  of  the  track  report.  Verify  that  the  platform  does 
not  artificially  increase  or  decrease  TQ  so  that  track 
correlation  gates  are  accurately  determined.  Verity  accurate 
and  consistent  TQ  reporting  over  the  LINK  Verify  proper 
decrement  of  TQ 

Treck  Management 

ST13  -  Verify  CEPN/LTN  pamng  consistency  across  all  Cus 

Track  Attribute 
Association 

ST  14  -  Verify  that  track  attributes  (TN,  IFF,  ID)  ere  correctly 
associated  and  maintained  on  each  object  of  interest 

CNA 
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DEP  Specific  ICC  Master  Requirements 


DEP  SPECIFIC  ICC  Matter  Requlrment* 

Function 

Subfunction 

Functional  Requirement 

Identification 

ID-DEP 

ID 

ID2  -  Vartfy  that  ID  drffaranca  protocols  are  processed  at  tha  forca 
laval  (ex  ID  conflict  pending  subsaquant  ID  conflict  raceived  whila 
panding) 

ID4  -  Vanfy  that  CEC  units  can  exchange  and  operate  with  COMP 

ID  doctnna 

IDS  -  Vartfy  that  CEC/LINK  ID  feedback  loop  design  issue  does  not 
prevent  proper  ID  managamant 

CDO 

ID7  -  Vanfy  that  “force  order"  and  change  data  order"  actions 
results  in  CDO  mes sagas  sant  on  both  LINK  and  CEC 

IDS  -Varrfy  tha  combat  systam  will  not  accept  any  ID  changas 
for  30  seconds  aftar  receipt  of  a  CDO 

SIAP 

ID  12  -  Evaluata  capability  of  tha  platform  whan  operating  with 
tha  Strika  Force  to  astablish  and  maintain  an  accurata  ID  for 
aach  trackad  object  (SIAP  ID  accuracy) 

ID13  -  Evaluata  capability  of  the  platform  whan  operating  with 
the  Strike  Force  to  maintain  a  clear  ID  for  each  object  such 
that  the  object  is  not  labaled  with  conflicting  ID  statas 
(SIAP  ID  clarity) 

Friendly  Force 
Deconflictlon 
FD-DEP 

12/15/19  Bit  IU  Mgmt 

FD2  Varify  correct  racaipt  and  display  at  combat  systam  of  C2 
and  Non-C2  PPLI  reports  (J2  2,2  3,2  4)  including  4  and  5  digit  PPLIs 

FD4  Vanfy  tha!  no  ID  or  IFF  diffarances  are  issuad  against  C2  or 

NorvC2  PPLI  reports 

FD6  Verify  translation  of  12,  15  and  19  bit  formats  "of  Pus 

and  Pus  used  as  LTNs 

NON-DEP  SPECIFIC  ICC  Matter  Requirements 

Force  Coordination 
FC-NONDEP 

Engagement 

Coordination 

FC1  Vanfy  proper  transmission,  recaipt  and  display  of  J9  0 
command  massages 

Engagement 

Statue 

FC3  -  Verify  proper  transmission,  receipt  and  display  of  J10  2 
anqaqamant  status 

CNA 
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Appendix  E 


•  ICC  Master  Requirements  Trace  Matrix  for  the  SSDS  10  DEV  08 
Event 


CNA 


This  appendix  contains  the  entire  example  requirements  trace  matrix  created  during 
the  BUR  for  the  SSDS  10  DEV  08  event. 
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REQOl 


Appendix  F 


FY09  DEP  Future  State  Timeline 


CNA 


This  appendix  contains  a  detailed  FY09  DEP  Future  State  timeline  for  multiple 
events  on  a  single  sheet. 


93 


Appendix  G 


DM&A  and  ICC  Rapid  Improvement  Event  Future  State  Diagram 


CNA 


This  appendix  includes  the  DM&A  and  ICC  Rapid  Improvement  Event’s  Future 
State  process  diagram. 
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